{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.2 深度学习基础及数学原理\n",
    "深度学习并没有想象的那么难，甚至比有些传统的机器学习更简单。所用到的数学知识也不需要特别的高深，本章将会一边讲解深度学习中的基本理论，一边通过动手使用PyTorch实现一些简单的理论，本章内容很多，所以只做一个简短的介绍\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.1 监督学习和无监督学习\n",
    "监督学习、无监督学习、半监督学习、强化学习是我们日常接触到的常见的四个机器学习方法：\n",
    "\n",
    "- 监督学习：通过已有的训练样本（即已知数据以及其对应的输出）去训练得到一个最优模型（这个模型属于某个函数的集合，最优则表示在某个评价准则下是最佳的），再利用这个模型将所有的输入映射为相应的输出。\n",
    "- 无监督学习：它与监督学习的不同之处，在于我们事先没有任何训练样本，而需要直接对数据进行建模。 \n",
    "- 半监督学习 ：在训练阶段结合了大量未标记的数据和少量标签数据。与使用所有标签数据的模型相比，使用训练集的训练模型在训练时可以更为准确。\n",
    "- 强化学习：我们设定一个回报函数（reward function），通过这个函数来确认是否越来越接近目标，类似我们训练宠物，如果做对了就给他奖励，做错了就给予惩罚，最后来达到我们的训练目的。\n",
    "\n",
    "这里我们只着重介绍监督学习，因为我们后面的绝大部们课程都是使用的监督学习的方法，在训练和验证时输入的数据既包含输入x,又包含x对应的输出y，即学习数据已经事先给出了正确答案。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.2 线性回归 （Linear Regreesion）\n",
    "线性回归是利用数理统计中回归分析，来确定两种或两种以上变量间相互依赖的定量关系的一种统计分析方法，运用十分广泛。其表达形式为y = w'x+e，e为误差服从均值为0的正态分布。 \n",
    "\n",
    "回归分析中，只包括一个自变量和一个因变量，且二者的关系可用一条直线近似表示，这种回归分析称为一元线性回归分析。如果回归分析中包括两个或两个以上的自变量，且因变量和自变量之间是线性关系，则称为多元线性回归分析。\n",
    "摘自[百度百科](https://baike.baidu.com/item/线性回归/8190345)\n",
    "\n",
    "简单的说：\n",
    "线性回归对于输入x与输出y有一个映射f，y=f(x),而f的形式为aX+b。其中a和b是两个可调的参数，我们训练的时候就是训练a，b这两个参数。\n",
    "\n",
    "下面我们来用pyTorch的代码来做一个详细的解释"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.4.1'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 引用\n",
    "# 注意，这里我们使用了一个新库叫 seaborn 如果报错找不到包的话请使用pip install seaborn 来进行安装\n",
    "import torch\n",
    "from torch.nn import Linear, Module, MSELoss\n",
    "from torch.optim import SGD\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面定义一个线性函数，这里使用 $y = 5x + 7$，这里的5和7就是上面说到的参数a和b，我们先使用matplot可视化一下这个函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7f15aa7e2550>]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzt3Xl4VPXd/vH3F0iAQFhCWAIhhH1LQDHsLigugAgial2KuJVq61NrWyGKC+6gtta2Lg+u2Fo3EhYFVETcFQUK2QgQIEAgJEAgCWTPfH9/ZHgufjRBYGZyZib367pyZTJzJufmzMydkzMnH4y1FhERCV6NnA4gIiK+paIXEQlyKnoRkSCnohcRCXIqehGRIKeiFxEJcip6EZEgp6IXEQlyKnoRkSDXxOkAAJGRkTY2NtbpGCIiAWXdunUHrLXtf245vyj62NhY1q5d63QMEZGAYozZeSrL6dCNiEiQU9GLiAQ5Fb2ISJBT0YuIBDkVvYhIkFPRi4gEORW9iEiQU9GLiDigtKKap1ZsIudQic/X5Rd/MCUi0pB8t+0AiUmp7CooIbptGNNGdPPp+lT0IiL1pKiskqeWb+KdH3cT2y6Md2eMYESPdj5fr4peRKQerMzI44HFqewvLufXF/Tgnov70Cykcb2sW0UvIuJDB46UM2dpOh+l5NKvUziv3JTAoOg29ZpBRS8i4gPWWpZs2MsjH6ZztLyaP17Sh19f0JPQJvV/DoyKXkTEy/YeLuWBxWl8npnP2TFteHrqIHp3DHcsj4peRMRLXC7Lv3/cxdwVmVS7LA9NHMD0UbE0bmQczaWiFxHxgh0HjjIrKYUfdxRwbq9Inroqnq4RYU7HAlT0IiIeqap28eo3O3hu5RZCmzTi6amDuCYhGmOc3Ys/nopeROQMZewtYlZSCql7Crl0QEceuzKOjq2aOR3rv/xs0RtjXgcmAvnW2jj3dRHAe0AskA1ca609ZGp+hD0PTABKgJuttet9E11ExBnlVdX84/MsXvpiG23CQnjhhiFMiO/kV3vxxzuV83zeBMadcF0isMpa2xtY5f4aYDzQ2/0xA3jJOzFFRPzDup2HuPxv3/D3z7OYdFZnVt5zAZcPivLbkodT2KO31n5ljIk94erJwBj35QXAF8As9/VvWWst8IMxpo0xJspam+utwCIiTiipqOKZTzbz5nfZdG7dnDdvGcqYvh2cjnVKzvQYfcdj5W2tzTXGHPvXdgF2H7dcjvs6Fb2IBKxvth4gMTmFnEOl3DSyGzPH9aNl08B5i9PbSWv73cXWuqAxM6g5vENMTIyXY4iIeK6wpJInlmfw/tocekS24P1fj2RY9winY522My36vGOHZIwxUUC++/ocoOtxy0UDe2v7Btba+cB8gISEhFp/GIiIOOXjtH08uCSNgqMV3DmmJ3eP7V1vQ8i87UyLfikwHZjr/rzkuOvvMsa8CwwHCnV8XkQCyf7imiFky1JzGRDVijduHkpcl9ZOx/LIqZxe+Q41b7xGGmNygIepKfj3jTG3AbuAa9yLL6fm1Mosak6vvMUHmUVEvM5aS/L6PTz6UQalFdXce1lfZpzfg5DGgf8f8Z3KWTfX13HT2FqWtcBvPQ0lIlKf9hwu5f7kVL7csp9zurVl3tRB9OrQ0ulYXhM4bxuLiHiZy2X515qdzFuRiQUemTSQaSO60cjhIWTepqIXkQZp2/4jJCal8FP2Ic7rHcmTU/xnCJm3qehFpEGprHbxytfb+etnW2ke0phnrxnM1CFd/PovWz2loheRBiNtTyGzklJI31vE+LhOPDJ5IB3C/W8Imbep6EUk6JVVVvP3z7fy8pfbaRsWyks3DmF8fJTTseqNil5Egtra7AJmJqWwff9RrjknmtmX96dNWKjTseqVil5EgtKR8iqe+TiTt37YSefWzXnr1mGc36e907EcoaIXkaDz5Zb93J+cyt7CUqaPjOXey/rSIoCGkHlbw/2Xi0jQOVxSwWMfbSJpfQ4927fgg1+PJCE28IaQeZuKXkSCworUXB5cks6hkgruurAXd13UK2CHkHmbil5EAlp+URkPLUnn4/R9xHVpxYJbhzKwc2APIfM2Fb2IBCRrLQvX5fDYRxmUVbmYNa4fvzqvO02CYAiZt6noRSTg7C4o4f5FqXy99QDDYiOYOzWeHu2DZwiZt6noRSRgVLssb32fzTOfbMYAj00eyI3Dg28Imbep6EUkIGTlFzMrKZV1Ow9xQZ/2PHlVPF3aNHc6VkBQ0YuIX6usdvG/X27jb6uyCGvamL9cO5gpZwf3EDJvU9GLiN9KzSlkZlIKm3KLuHxQFHOuGEj78KZOxwo4KnoR8TtlldX89bOtvPL1dtq1COV/p53DZQM7OR0rYKnoRcSvrNl+kMTkVHYcOMovErpy/+X9ad08xOlYAU1FLyJ+obiskqc/3sw/f9hJ14jmvH37cEb3inQ6VlBQ0YuI41Zvzmd2ciq5RWXcOro7f7qsD2Ghqidv0ZYUEcccOlrBYx9lkPyfPfTu0JKkO0cxJKat07GCjopeROqdtZZlqbk8vCSdwtJKfje2N7+9sCdNm2gImS+o6EWkXuUVlfHA4jRWZuQxKLo1/7p9OP2jWjkdK6ip6EWkXlhreX/tbh5ftomKKhf3T+jHraM1hKw+qOhFxOd2HSwhMTmF77YdZHj3COZNHURsZAunYzUYKnoR8Zlql+XN77J59pPNNG5keGJKHNcPjdEQsnqmohcRn9iSV8zMhSls2H2Yi/p14IkpcUS11hAyJ6joRcSrKqpcvPTFNv6xeistmzbh+evOYtLgzhpC5iAVvYh4zcbdh5mVlELmvmImDe7Mw1cMoF1LDSFzmopeRDxWWlHNc59t4dWvt9MhvBmv3pTAxQM6Oh1L3DwqemPMPcDtgAVSgVuAKOBdIAJYD0yz1lZ4mFNE/NT32w5yX3IK2QdLuH5YDPdN6EerZhpC5k/O+ARWY0wX4HdAgrU2DmgMXAfMA56z1vYGDgG3eSOoiPiXorJK7l+UyvWv/IAF/v2r4Tx1VbxK3g95euimCdDcGFMJhAG5wEXADe7bFwBzgJc8XI+I+JFVm/KYvSiN/OIyfnVed/5wSV+ah2p8gb8646K31u4xxjwL7AJKgU+BdcBha22Ve7EcoIvHKUXELxw8Us4jH2awdONe+nYM5+Vp53BW1zZOx5KfccZFb4xpC0wGugOHgQ+A8bUsauu4/wxgBkBMTMyZxhCRemCtZenGvTzyYQbFZZXcc3Ef7hzTk9AmGl8QCDw5dHMxsMNaux/AGJMMjALaGGOauPfqo4G9td3ZWjsfmA+QkJBQ6w8DEXFebmEpDyxKY1VmPoO7tuHpqYPo2ync6VhyGjwp+l3ACGNMGDWHbsYCa4HVwNXUnHkzHVjiaUgRqX8ul+Xdn3bz1PJNVLpcPHB5f24Z3Z3GGl8QcDw5Rr/GGLOQmlMoq4D/ULOHvgx41xjzuPu617wRVETqT/aBoyQmp/DD9gJG9mjH3KnxdGunIWSByqOzbqy1DwMPn3D1dmCYJ99XRJxRVe3ijW+z+fPKzYQ0asTcq+L5xdCuGl8Q4PSXsSICQOa+ImYtTGFjTiEX9+/I41fG0al1M6djiReo6EUauPKqal5YvY0XV2fRunkIf7/+bCYOitJefBBR0Ys0YP/ZdYhZSSlsyTvClLO78ODEAUS0CHU6lniZil6kASqpqOLPn27h9W930KlVM16/OYGL+mkIWbBS0Ys0MN9lHSAxOZVdBSX8ckQMs8b1I1zzaYKail6kgSgsreSp5Zt496fddI9swbszRjCiRzunY0k9UNGLNACfpu/jgcVpHDhSzq8v6ME9F/ehWYiGkDUUKnqRIHbgSDlzlqbzUUou/TqF8+r0BAZFawhZQ6OiFwlC1loWb9jDIx9mUFJezR8v6cMdY3oS0lhDyBoiFb1IkNl7uJTZi1JZvXk/Z8fUDCHr3VFDyBoyFb1IkHC5LG//uIt5KzKpdlkemjiA6aNiNYRMVPQiwWD7/iMkJqXyY3YB5/aK5Kmr4ukaEeZ0LPETKnqRAFZV7eLVb3bw3MotNG3SiKevHsQ150RrfIH8f1T0IgEqY28RM5M2kraniMsGduSxyXF0aKUhZPLfVPQiAaa8qpp/fJ7FS19so01YCC/eOITxcZ20Fy91UtGLBJB1OwuYlZRKVv4RrhrShQcvH0BbDSGTn6GiFwkAR8ureOaTzSz4PpvOrZvz5i1DGdO3g9OxJECo6EX83Ndb93Nfcio5h0qZPrIb947rR8umeunKqdOzRcRPFZZU8viyDD5Yl0OP9i344I6RDI2NcDqWBCAVvYgf+jhtHw8uSaPgaAW/GdOT343trSFkcsZU9CJ+JL+4jDlL01meuo8BUa144+ahxHVp7XQsCXAqehE/YK0laf0eHvsog9LKau69rC8zzu+hIWTiFSp6EYflHCrh/kVpfLVlPwnd2jJ36iB6dWjpdCwJIip6EYe4XJZ//rCTeR9nAvDIpIFMG9GNRhpCJl6mohdxwLb9R5i1MIW1Ow9xfp/2PDkljui2GkImvqGiF6lHldUu5n+1nedXbaV5SGOevWYwU4d00fgC8SkVvUg9SdtTyMyFKWTkFjEhvhNzJg2kQ7iGkInvqehFfKyssprnV21l/lfbaRsWysu/HMK4uCinY0kDoqIX8aGfsguYtTCF7QeOcs050Txw+QBah4U4HUsaGBW9iA8cKa/i6Y8zeev7nUS3bc4/bxvGeb3bOx1LGiiPit4Y0wZ4FYgDLHArsBl4D4gFsoFrrbWHPEopEkC+3LKf+5NT2VtYys2jYrn3sr600BAycZCnf3b3PPCxtbYfMBjYBCQCq6y1vYFV7q9Fgt7hkgr+8P4Gpr/+I81CGrHwjpHMmTRQJS+OO+NnoDGmFXA+cDOAtbYCqDDGTAbGuBdbAHwBzPIkpIg/s9ayIm0fDy1J43BJJXdd2Iu7LuqlIWTiNzzZ1egB7AfeMMYMBtYBdwMdrbW5ANbaXGOM/ncECVr5RWU8uCSNT9LziOvSigW3DmNgZw0hE//iSdE3AYYA/2OtXWOMeZ7TOExjjJkBzACIiYnxIIZI/bPW8sG6HB7/KIPyKheJ4/tx+7ndaaIhZOKHPCn6HCDHWrvG/fVCaoo+zxgT5d6bjwLya7uztXY+MB8gISHBepBDpF7tLijhvuRUvsk6wLDYCOZOjadHew0hE/91xkVvrd1njNltjOlrrd0MjAUy3B/Tgbnuz0u8klTEYdUuy1vfZ/P0x5tpZOCxK+O4cViMhpCJ3/P0dID/Ad42xoQC24FbqDmT531jzG3ALuAaD9ch4ris/GJmLkxh/a7DjOnbniemxNOlTXOnY4mcEo+K3lq7AUio5aaxnnxfEX9RWe3i5S+28ffPswhr2pjnfjGYK8/SEDIJLDrBV6QOqTmF3LtwI5n7ipk4KIo5kwYS2bKp07FETpuKXuQEZZXVPPfZFl75ajuRLZsyf9o5XDqwk9OxRM6Yil7kOGu2HyQxOZUdB45y3dCu3DehP62bawiZBDYVvQhQXFbJvI8z+dcPu+ga0Zy3bx/O6F6RTscS8QoVvTR4qzPzuX9RKvuKyrjt3O788dI+hIXqpSHBQ89mabAKjlbw6IfpLN6wl94dWpJ05yiGxLR1OpaI16nopcGx1vJRSi5zlqZTWFrJ78b25rcX9qRpEw0hk+CkopcGJa+ojNmL0vhsUx6Dolvz9q+G069TK6djifiUil4aBGst7/20myeWb6KiysXsCf25ZXSshpBJg6Cil6C362AJickpfLftIMO7RzBv6iBiI1s4HUuk3qjoJWhVuyxvfLuDZz/dTJNGjXhySjzXDe2qIWTS4KjoJSht3lfMzKQUNu4+zEX9OvDElDiiWmsImTRMKnoJKhVVLl78IosXVmcR3iyE5687i0mDO2sImTRoKnoJGht3H2bmwhQ25xUz+azOPDRxAO00hExERS+Br7Simr+s3Mxr3+ygQ3gzXr0pgYsHdHQ6lojfUNFLQPtu2wHuS05l58ESbhgeQ+L4frRqpiFkIsdT0UtAKiqr5Knlmbzz4y66tQvj378azqieGkImUhsVvQSczzLymL04lf3F5cw4vwf3XNyH5qEaXyBSFxW9BIyDR8p55MMMlm7cS79O4cyflsDgrm2cjiXi91T04vestSzduJc5S9M5Ul7FPRf34c4xPQltovEFIqdCRS9+LbewlAcWpbEqM5+zurbh6asH0adjuNOxRAKKil78kstleeenXTy1PJMql4sHLu/PLaO701jjC0ROm4pe/M6OA0dJTEphzY4CRvVsx9yrBhHTLszpWCIBS0UvfqOq2sXr3+7gz59uIbRJI+ZNjefahK4aXyDiIRW9+IVNuUXMSkohJaeQSwZ05PEr4+jYqpnTsUSCgopeHFVeVc0Lq7fx4uosWjcP4R83nM3l8VHaixfxIhW9OGb9rkPMWpjC1vwjTDm7Cw9NHEDbFqFOxxIJOip6qXclFVU8+8kW3vhuB51aNeONm4dyYb8OTscSCVoqeqlX32YdIDE5hd0FpfxyRAyzxvUjXEPIRHxKRS/1orC0kieXbeK9tbvpHtmC92aMYHiPdk7HEmkQVPTic5+m7+OBxWkcPFrBHRf05PcX96ZZiIaQidQXj4veGNMYWAvssdZONMZ0B94FIoD1wDRrbYWn65HAs7+4nDkfprMsJZf+Ua14bfpQ4qNbOx1LpMHxxlSou4FNx309D3jOWtsbOATc5oV1SACx1pK8PodLnvuSlel5/OnSPiy9a7RKXsQhHhW9MSYauBx41f21AS4CFroXWQBc6ck6JLDsOVzKLW/+xB/e30iPyBYsv/tc7rqoNyGNNWlSxCmeHrr5KzATODZOsB1w2Fpb5f46B+ji4TokALhclrfX7GTuikxcFh6+YgA3jYzVEDIRP3DGRW+MmQjkW2vXGWPGHLu6lkVtHfefAcwAiImJOdMY4ge27z9CYlIqP2YXcF7vSJ6cEk/XCA0hE/EXnuzRjwYmGWMmAM2AVtTs4bcxxjRx79VHA3tru7O1dj4wHyAhIaHWHwbi36qqXbzy9Q6e+2wLzZo04pmrB3H1OdEaXyDiZ8646K219wH3Abj36P9krb3RGPMBcDU1Z95MB5Z4Iaf4mYy9RcxM2kjaniIuG9iRxybH0UFDyET8ki/Oo58FvGuMeRz4D/CaD9YhDimrrOYfn2fx8pfbaBMWyks3DmF8fJTTsUTkJLxS9NbaL4Av3Je3A8O88X3Fv6zbWcDMhSls23+UqUOieXBif9qEaQiZiL/TX8bKzzpaXsUzn2xmwffZdG7dnAW3DuOCPu2djiUip0hFLyf11Zb93Jecyt7CUm4a0Y17x/WjZVM9bUQCiV6xUqvCkkoeW5bBwnU59Gjfgvd/PZKhsRFOxxKRM6Cil//ycVouDy5Jp+BoBb8Z05PfjdUQMpFApqKX/5NfXMbDS9JZkbaPAVGteOPmocR10XwakUCnohestSxcl8PjyzZRWlnNvZf1Zcb5PTSfRiRIqOgbuN0FJdy/KJWvtx4goVtb5k4dRK8OLZ2OJSJepKJvoFwuy1vfZ/P0J5sxwKOTB/LL4d1opCFkIkFHRd8AZeUfITEphbU7D3F+n/Y8OSWO6LYaQiYSrFT0DUhltYv5X23n+c+20jy0MX++ZjBXDemiIWQiQU5F30Ck7Slk5sIUMnKLmBDfiUcmxdE+vKnTsUSkHqjog1xZZTXPr9rK/K+2E9EilJd/OYRxcRpCJtKQqOiD2E/ZBcxamML2A0e5NiGa2RMG0DosxOlYIlLPVPRB6Eh5FU9/nMlb3+8kum1z/nXbcM7tHel0LBFxiIo+yKzenM/s5FRyi8q4ZXQsf7q0Ly00hEykQVMDBIlDRyt47KMMkv+zh14dWrLwjlGc062t07FExA+o6AOctZblqft4eGkah0sq+Z+LenHXRb1o2kRDyESkhoo+gOUXlfHA4jQ+zcgjvktr3rp1OAM6t3I6loj4GRV9ALLW8sHaHB5blkFFlYv7xvfjtnO700RDyESkFir6ALO7oIT7klP5JusAw7pHMPeqeHq01xAyEambij5AVLssC77L5plPNtO4keHxK+O4YViMhpCJyM9S0QeArXnFzExK4T+7DjOmb3uenBJP5zbNnY4lIgFCRe/HKqpcvPzlNv7xeRYtmjbmr784i8lnddYQMhE5LSp6P5WSc5iZC1PI3FfMFYM78/AVA4hsqSFkInL6VPR+pqyymudWbuGVr7fTPrwpr9yUwCUDOjodS0QCmIrej/yw/SCJSSlkHyzh+mFdSRzfn9bNNYRMRDyjovcDxWWVzF2RydtrdhETEca/bx/OqF4aQiYi3qGid9jnmXnMXpRGXlEZt5/bnT9c2oewUD0sIuI9ahSHFByt4NEP01m8YS+9O7TkxTtHcXaMhpCJiPep6OuZtZYPU3KZszSd4rJK7h7bm99c2FNDyETEZ1T09WhfYc0Qss825TE4ujXzrh5Ov04aQiYivnXGRW+M6Qq8BXQCXMB8a+3zxpgI4D0gFsgGrrXWHvI8auCy1vLuT7t5ctkmKl0uZk/oz63ndqexxheISD3wZI++CvijtXa9MSYcWGeMWQncDKyy1s41xiQCicAsz6MGpp0Hj5KYlMr32w8yokcEc68aRGxkC6djiUgDcsZFb63NBXLdl4uNMZuALsBkYIx7sQXAFzTAoq92Wd74dgfPfrqZkEaNeHJKPNcN7aohZCJS77xyjN4YEwucDawBOrp/CGCtzTXGdKjjPjOAGQAxMTHeiOE3Nu+rGUK2cfdhxvbrwONT4ohqrSFkIuIMj4veGNMSSAJ+b60tOtWBW9ba+cB8gISEBOtpDn9QUeXixS+yeGF1FuHNQvjb9WdzxaAoDSETEUd5VPTGmBBqSv5ta22y++o8Y0yUe28+Csj3NGQg2LD7MLMWprA5r5jJZ3Xm4SsGEtEi1OlYIiIenXVjgNeATdbavxx301JgOjDX/XmJRwn9XGlFNX/+dDOvf7uDDuHNeG16AmP7awiZiPgPT/boRwPTgFRjzAb3dfdTU/DvG2NuA3YB13gW0X99t+0AiUmp7Coo4YbhMSSO70erZhpCJiL+xZOzbr4B6jr4PPZMv28gKCqr5Knlm3jnx910axfGO78awcie7ZyOJSJSK/1l7Gn6LCOP2YtT2V9czozze3DPxX1oHqrxBSLiv1T0p+jgkXLmfJjBhxv30q9TOPOnJTC4axunY4mI/CwV/c+w1rJkw14e+TCdI+VV/OGSPtxxQU9CmzRyOpqIyClR0Z/E3sOlPLA4jc8z8zmraxuevnoQfTqGOx1LROS0qOhr4XJZ/v3jLuauyKTaZXlw4gBuHhWrIWQiEpBU9CfYceAoiUkprNlRwOhe7XhqyiBi2oU5HUtE5Iyp6N2qql289s0O/rJyC6FNGjFvajzXJnTV+AIRCXgqemBTbhGzklJIySnkkgEdefzKODq2auZ0LBERr2jQRV9eVc0Ln2fx4hfbaBMWwgs3DGFCfCftxYtIUGmwRb9u5yFmJaWQlX+Eq87uwoMTB9BWQ8hEJAg1uKIvqajimU828+Z32US1asYbtwzlwr61jswXEQkKDarov9l6gMTkFHIOlTJtRDdmjutLuIaQiUiQaxBFX1hayRPLMnh/bQ7dI1vw3owRDO+hIWQi0jAEfdF/kr6PBxencfBoBXeO6cndY3vTLERDyESk4Qjaot9fXM6cpeksS82lf1QrXps+lPjo1k7HEhGpd0FX9NZaktfv4dGPMiitqObey/oy4/wehDTWEDIRaZiCquj3HC7l/uRUvtyynyExNUPIenXQEDIRadiCouhdLsu/1uxk3opMLDDnigFMG6khZCIiEARFv23/ERKTUvgp+xDn9Y7kySnxdI3QEDIRkWMCuujf/2k3DyxJo1mTRjxz9SCuPida4wtERE4Q0EXfvX0LxvbrwCOTB9IhXEPIRERqE9BFPzQ2gqGxEU7HEBHxazrnUEQkyKnoRUSCnIpeRCTIqehFRIKcil5EJMip6EVEgpyKXkQkyKnoRUSCnLHWOp0BY8x+YOcZ3j0SOODFON6iXKdHuU6fv2ZTrtPjSa5u1tr2P7eQXxS9J4wxa621CU7nOJFynR7lOn3+mk25Tk995NKhGxGRIKeiFxEJcsFQ9POdDlAH5To9ynX6/DWbcp0en+cK+GP0IiJycsGwRy8iIicRMEVvjBlnjNlsjMkyxiTWcntTY8x77tvXGGNi6yFTV2PMamPMJmNMujHm7lqWGWOMKTTGbHB/POTrXO71ZhtjUt3rXFvL7cYY8zf39koxxgyph0x9j9sOG4wxRcaY35+wTL1tL2PM68aYfGNM2nHXRRhjVhpjtro/t63jvtPdy2w1xkz3caZnjDGZ7sdpkTGmTR33Pelj7qNsc4wxe457vCbUcd+Tvn59kOu94zJlG2M21HFfn2yzurrBseeXtdbvP4DGwDagBxAKbAQGnLDMb4CX3ZevA96rh1xRwBD35XBgSy25xgAfObDNsoHIk9w+AVgBGGAEsMaBx3QfNecBO7K9gPOBIUDacdc9DSS6LycC82q5XwSw3f25rftyWx9muhRo4r48r7ZMp/KY+yjbHOBPp/BYn/T16+1cJ9z+Z+Ch+txmdXWDU8+vQNmjHwZkWWu3W2srgHeByScsMxlY4L68EBhrfPwfyFprc621692Xi4FNQBdfrtOLJgNv2Ro/AG2MMVH1uP6xwDZr7Zn+oZzHrLVfAQUnXH3882gBcGUtd70MWGmtLbDWHgJWAuN8lcla+6m1tsr95Q9AtDfWdbrq2F6n4lRevz7J5e6Aa4F3vLW+U8xUVzc48vwKlKLvAuw+7usc/rtQ/28Z94uiEGhXL+kA96Gis4E1tdw80hiz0RizwhgzsJ4iWeBTY8w6Y8yMWm4/lW3qS9dR94vPie11TEdrbS7UvFiBDrUs4+S2u5Wa38Rq83OPua/c5T6s9HodhyKc3F7nAXnW2q113O7zbXZCNzjy/AqUoq9tz/zE04VOZRmfMMa0BJKA31tri064eT01hycGA38HFtdHJmC0tXYIMB74rTEm3xNmAAACcklEQVTm/BNud3J7hQKTgA9qudmp7XU6HNl2xpjZQBXwdh2L/Nxj7gsvAT2Bs4Bcag6TnMix5xpwPSffm/fpNvuZbqjzbrVc59H2CpSizwG6Hvd1NLC3rmWMMU2A1pzZr5mnxRgTQs0D+ba1NvnE2621RdbaI+7Ly4EQY0ykr3NZa/e6P+cDi6j59fl4p7JNfWU8sN5am3fiDU5tr+PkHTuE5f6cX8sy9b7t3G/ITQRutO4DuSc6hcfc66y1edbaamutC3iljnU68lxz98BVwHt1LePLbVZHNzjy/AqUov8J6G2M6e7eG7wOWHrCMkuBY+9OXw18XtcLwlvcx/9eAzZZa/9SxzKdjr1XYIwZRs02P+jjXC2MMeHHLlPzZl7aCYstBW4yNUYAhcd+pawHde5lObG9TnD882g6sKSWZT4BLjXGtHUfqrjUfZ1PGGPGAbOASdbakjqWOZXH3BfZjn9fZ0od6zyV168vXAxkWmtzarvRl9vsJN3gzPPL2+82++qDmrNEtlDz7v1s93WPUvPkB2hGzaGALOBHoEc9ZDqXml+pUoAN7o8JwB3AHe5l7gLSqTnT4AdgVD3k6uFe30b3uo9tr+NzGeAF9/ZMBRLq6XEMo6a4Wx93nSPbi5ofNrlAJTV7UbdR877OKmCr+3OEe9kE4NXj7nur+7mWBdzi40xZ1ByzPfYcO3Z2WWdg+cke83rYXv90P39SqCmxqBOzub/+r9evL3O5r3/z2PPquGXrZZudpBsceX7pL2NFRIJcoBy6ERGRM6SiFxEJcip6EZEgp6IXEQlyKnoRkSCnohcRCXIqehGRIKeiFxEJcv8Pye3TH8lSsHMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(0,20,500)\n",
    "y = 5*x + 7\n",
    "plt.plot(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我生成一些随机的点，来作为我们的训练数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.rand(256)\n",
    "noise = np.random.randn(256) / 4\n",
    "y = x * 5 + 7 + noise\n",
    "df = pd.DataFrame()\n",
    "df['x'] = x\n",
    "df['y'] = y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在图上显示下我们生成的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/jinbo/anaconda3/lib/python3.5/site-packages/scipy/stats/stats.py:1713: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.\n",
      "  return np.add.reduce(sorted[indexer] * weights, axis=axis) / sumval\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvqOYd8AAAIABJREFUeJzs3XmAZHdZ6P3v7yy1dC29THfPPpmZZEI2wCRjSF7COCwXoyhLjMq8oqjEzEUwXu6FG16FeI0bedXXKwiYEbiiaASDQPRKFG4YB3BGyEIgyyST6ZlklvT03rVXneX3/nGqqquqq7qr9+7p56OhmepTp04CPP3r5/f8nkdprRFCCLH8jJV+ACGEWK8kAAshxAqRACyEECtEArAQQqwQCcBCCLFCJAALIcQKkQAshBArRAKwEEKsEAnAQgixQqyVfoB23HLLLfqhhx5a6ccQQoh2qXYuWhMr4JGRkZV+BCGEWHRrIgALIcTFSAKwEEKsEAnAQgixQiQACyHECpEALIQQK0QCsBBCrBAJwEIIsUIkAAshxAqRACyEECtkTRxFFkKIhTh8fIj7jgxwZjzH9u4ODu7bzf4r+lf6sWQFLIS4uB0+PsTdDz7FULpAV9RmKF3g7gef4vDxoZV+NAnAQoiL231HBrBNRUfIQqngq20q7jsysNKPJgFYCHFxOzOeI2qbda9FbZOz47kVeqIpEoCFEBe17d0d5B2v7rW847Gtu2OFnmiKBGAhxEXt4L7dOJ4mV3LROvjqeJqD+3av9KNJABZCXNz2X9HPPW++mv5EhMm8Q38iwj1vvnpVVEFIGZoQ4qK3/4r+VRFwG8kKWAghVogEYCGEWCESgIUQYpForRlKF9q+XgKwEEIsAt/XXEgV+ZcnB9t+j2zCCSHWncXuDeH5msFUgfv/40U+/o3n+fmbdrb1PlkBCyHWlcXuDVFyfc6O5/ifX3uOP/vG8+g5vFcCsBBiXVnM3hAFx+P0SIYPf/lJ/v7RswC8+rINbb9fUhBCiHXlzHiOrqhd91rUNjlxIcWBQ8faTkvkSi7PDab5jS89ydMvpQC49dqtvHv/pW0/iwRgIcS6Eg+ZPD+cwfM1IdOgNx6m6Hqki960tMQ90DQIpwoOT7w4wQf/4Qecm8ijgHfvv5Sf2bud/mS47WeRFIQQYt04fHyI0WwJ19MowPF8zk3kGckU6YnZbaUlxrMl/u3ZId57/+Ocm8gTsgx+6yev4u0/vIPNXRE6Qu2va2UFLIRYN+47MkAyahMLWwyni5Q8H8tQeBo2xOpXrs1aVg6ni/zv75/n9796nJLrk4xY/N7bruHaHd1sSkawzLmtaSUACyHWjUr+VylFIhLkgbXWnBjKkHe8utVrbctKrYMa3786eppPHj6JBrZ2RfmDW6/h8o1J+hNhDEPN+XkkAAshLlqN9b6JsNU00O7ujZEteeRKLlHbJO941ZaVnq85N5Hnf37tOf7h8XMAXLU5we++9Rou2RBjQ7z9nG8jCcBCiItSpd7XNlV1Y20y76CAbNElXXApuj6moXjP/kt5xbYu7jsywNnxHNvKVRCv3tPLwHCG//GPT/Ht50cBeM2eXn7zx69kS3eUZMSe+SFmIQFYCHFRqq33BapfiyWX4ayD6/tYSqG15k8fPsGevjgf/LErq1UPBcfj6fOTfPAffsAzL6UB+Onrt/Hu/ZeyuTNKNGQ2/+A5WLIqCKXUZ5RSQ0qpJ2te+0Ol1HGl1PeVUl9SSnUt1ecLIda3VrPghrMO27qjbO/uQKNQSmEqxemxXPVEXK7k8t1TY7z7bx7jmZfSKOC9r72UO1+/h23dHYsSfGFpy9D+Eril4bWvAddorV8BPAf8P0v4+UKIdejw8SEOHDrGcLrI88MZUnmn+r3KbLiobTKcLqIUGEphKIXna2xT8fHDz/N/nhniPX/7GOcnCoQsg99+89UceNUlbOmKErIWL2wuWQpCa31EKbWz4bV/rfnjMeC2pfp8IcT6U5v33ZQMc26iwLmJPKCxTAPH0+zujZF3PEpekP8F0BpCpoFtKk5cSPNfv/A9HE/TGbX5vbdeww27euhLhFFq7pUOM1nJHPAvA59v9U2l1B3AHQA7duxYrmcSQqwxtZUOqbxDR8ikMxoBQCnF4GSBwVSR63Z0Vwdx3v3gU5hK4fvBgQwfTVc0xOBkgYm8C8C27ih/cOvLuXpLJz2x0JI8+4oEYKXUbwIu8DetrtFaHwIOAezdu3cuDYaEEOtEY6XDS5N58o5H2DJJRm0SEZt42GIy73Bw3+5qoI6HTPoSYc5PFrAMTV8sxGTBIVMMUhRXb0nye2+9hkv749V64aWw7EeRlVLvBH4C+DmttQRWIcS8NXY2i1jB5thIpli9Ju94xMNWXQtKx9copfi1117KlZuSjGangu++y3v5k595JS/blFzS4AvLvAJWSt0C3AX8iNY6N9v1Qggxk8bOZn2JMOfG8xQcj5NDaYqej2UY9MVswiGrriQtW3T4+jNDuL6m6PpAUGb23tddxubOxd1sa2XJArBS6n5gP9CrlDoL/BZB1UMY+Fo5mX1Ma/2fl+oZhBAXt+3dHQylC9XAmojYJCIO43mHkqeJWCaJiMVL6SLbuqYCqq81ng/PvJTC02AoeO9rL+Nnb9jBpmSkujm31JayCuJAk5c/vVSfJ4RYfw7u283dDz5Vd4Q4W/LYmAjTl4hUr5vIOVxIF0lGQ/i+ZrLgcH4ij68hbBl86E1X8qNXb1qSSoeZSDtKIcSatf+Kfu5589X0JyJM5h36ExHiYZPehv4MG5NhHE+TLjiMZoucHQ+CrwIu6emgK2rTn4wsa/AFOYoshFjj9l/RX9c0/cChYwylC3i+rracNJWiPx6i5PoMpoINOlPB9p4oJc/nD//1ORIRe0GDOedDArAQ4qJx+PgQZ8aynJ0oVF8zFXhaM5ZzqpttIctge3eUiGViGIpcyeW+IwMSgIUQopWZxskfPj7E+x94gomcU/ceT4NtqmrwDZeDb9g2Mcoph2bN15eD5ICFEGvCbOPkP/LVZxjLlnD96ccLHC94LRYyuWpTAl/ravBN5R2eH84wlC5y4NCxeY+nnw9ZAQsh1oRm7SUrqQOAE8MZmsTeKtMIZsDlHY9UwUUphev5nCunK7Z2RWYdxrnYZAUshFgTzozncD2fgeEMxwdTDAxncD2fs+O5IDgbBjMVMXh+UPUwWQiastuGYjBVxDIU27qjJKOhGYdxLgUJwEKINSERtjg3UcD1NaahcH3NuYkC8bDFmfEcG5PhGQNayFSgFKMZB8/XnJ+cupfWkC44DAxneGE0y2Mvji9LKkICsBBiTai2jtE1f5Vf314enhkLN8+qqvK/GkqhCSoiciWPsKkolUfTnxnL4fpBblgp6vLLS0UCsBBiTciUPLZ2RbBMhac1lqnY2hUhW/J4+97tDGdKpApBK0nbUCiCwKtUEKsdz8fzNV45URy2DPqTEXxf4/oaT0PJ9XF9zcZEZFlSEbIJJ4RYEyp9H3b3xauvDafzpAsuH/iH71Mql5lFLAPTgJ6wTaboUXQ8XKaCsCYIzL3xMFoHPYOpacyolrE0TVbAQog14eC+3TieJldy0VoznM5zIVWi4PrV4BsLm/zeW6+hqyNEXyJCbzyMMhSWmloJKwWdUYtk1GYkU8RUqrpSjtgmllKMZIrkHY9t5dTGUpEALIRYE2r7PkzkSkzmXZQKqhsA+hNheuNhHnjsHDt6grFDyajNls7g0IWpFMmIxftev4dkNESu5FLyfDTBRpypFL7WoIL2lI6nqxM0loqkIIQQq1rj6bd3vXonmZLH+z7/veqKdnMygqkUgxN5XhjNYRsKFGyIheiNh7FMheNp7nnz1ey/op9XbOviviMDnB3Po4AtXUHntOF0kYLrEwtZ1WuXkloLQyn27t2rH3nkkZV+DCHEMqsdORS1TbIll/GsQ7rg4mmNoYLZbVrDuYmpDme2GUw5NgxFZ8Riz8Zk3bHlVvfPO15doF6AttqqyQpYCLFq1K524yGTF8fylDyfsGXQ3WGTd3wmymPm+xNBy0nLUFxIFaun4DSUy8nAVIo9G5Pcf8eNTT9v/xX93APl1XCObQ39JZaaBGAhxIqpBNwTQ2lyRa88UNMgETEZLB+UsA0oOR7nJrxK6S8hU/HZd93A+bE8n/rWKU6N1Fcr+Lr8l+/PWsnQ2M5yOckmnBBiRVR+/T89mmEy55BzvPLq1WckE6xyFeD4VMvIADpCJldv6eSKjQlef9VG7r/jRhIRq3q9Klc8VF5Y6kqGhZAVsBBiUczUKrKZSnOd0YyLYSiUr9EEK9dKGqGislWlgK6oza+/fk/d9IqQZZT7/oJq2NZa6kqGhZAVsBBiwWZrFdnMmfEcUduk5Pl1q9ZWHc0UEA+b/P7bXj4tsO/pT9CfDBOpmWQcMhUv25hYsfRCOyQACyEWrLZVpFJq1q5ih48Pkco7PDOYwvM1rqexjNbhyFBBZcPHDlzXNKAe3Lcb2zTZ2h3l6i1JdvfF2NQZ5a5brli0v8elIAFYCLFgldVsrVZHeSur5Y6QWW184/oaz/dpNg3eVMHx4mTEarmabTaccznqeBdKcsBCiDlrzPcmwhZ5x6s2SwcYzRbJFj1uvvfhupxwZbXcGY0QtkxGMkUKjlcdEV+ozG0zDbZ1R+kIBfW5/TVj5ptZyWqG+ZIALISYk9rDC5V872TeqZ48iNomo9kiQ+kSffFQXU74HuC5CykKjk/J8wmZBr3xMFHb4IWxHHlnKvhuTISI2kb1cMRq3kybLwnAQog5aTYaCIIWkN2xMGfHc2SLHn3xoCFO5ZpcyeUjX32GTNHD1xpTKVxPc3Y8h2Go6ty2N161kVuu3sjfP3puRQ5HLCcJwEKIOTkznqMrate9FrVNJvMOD70vOHF2870PV69JFxyG00WKroeng8GY2aKPw1S5g1cOvr9w0yX8tzdeTmc0xK3Xb1+mv6OVIwFYCDEnlb68tfnextaNlWs8X3N+ooBSYKhgjFCm6GEAjdVm//1HX8Yvvnpn3X0vdlIFIYSYk8a+vLmSOy1HW7lmcLIA6Mr/V/PEfsM9I5bBu16za10FX5AVsBBijtppYFO55uDnHkUT1PD2xiO8ODa9LM1UEAsZhC1z2veameuJu9VMArAQYs6alXw1C4zX7eiupis8369OM66IWIrOqM3uvkRbn9usAqNSXbEWg7AEYCHEgrUKjLddt5UHHjvH4GSOsaxDea8NU8Hu3hg+zKnErFkFxnC6wJ1/9zjJqL3mVsSSAxZCLFiro8hHB8b4T1f2M5KZCr62qTAUjOdKcz6x1njiLpV3GM2WyJbctntQrCayAhZCLFir0rSB4QyPnxmvVjwYCiwFXfEwOzfEWzZKb6WxAmMkUwQgYpnVwJ8rudx3ZGBNrIIlAAshmprLZldjYNRac24ix0TOrQbfyqggH8VIuoTjpee8oXZw327ufvApciWXqG1SdH0U0FeejgHLM05+sUgKQoh16vDxIQ4cOsbN9z7MgUPH6n5tn2t7ydrSNN/3OTOeY7wm+EK5x6+nMVQwMDNX8ubcwrKx6U5HyKQ3ESIRmVp9L8c4+cUiQzmFWIdmG0Z54NCxaYctciWX/kRkWtqgsop97kKKkuvj6SC4tmIpUIZCa80lG2J1n3FmLEu6GLw3FjK5/eZd3PmGy+f997GCZCinEKK5ZtUEtbnTVjndyq/2tbPc0gWX7g6b/kSYcxMFckUXCCKQpcCjvsm6q8EuL/xqN9QGJ/NM5IP3hi1F3vH404efB2gZhFd6qOZCLVkAVkp9BvgJYEhrfU35tZ8G/gdwJXCD1lqWtUKsgNoAm8o7jGSKlDyfs+N5Dh8fmvG4ce2qM1d08bVmOF1kMu9Uu5mZKhgjZJoGSkPJqz/7pgn6AI9kitWGPaPZEhBs1BnKwFDBfLhPfevUjKvgtdiGsmIpc8B/CdzS8NqTwK3AkSX8XCHELLZ3d5B3PFJ5h/OTeVxPBwMtgbsffIqbdve0PG5cu3p2fI1C42uqwTcZseiMmNiWgac1jV0fDAXbuqP0xkOM5xxyJZdUvjQ1Vl6DV/6DoSA7QzpjrVuyAKy1PgKMNbz2jNb62aX6TCFEeyqbZhfShSBZWf6XTZ2Rav1uqwkTtbW4poKSNxViE2GLguOSdTR9MRtfa0pefQCuJEc3xMIkIha2oTg7ka+7xvF8PD8I7LFQe0eU1yLJAQuxzlTyt9miQ9EJxgBFbIO+RJhExEbroEdvq1/tK+kJ368PrqYKNuqUUmztDGOZBooiIXOq169lKAxDMZwuYhqKPf3BEeSdviaddxjKlKrBvOQFR5dvv3nXkv8zWSmrNgArpe4A7gDYsWPHCj+NEGtPsxpboJq/3dwZJVvycD1dDb5QX8bV7B53vGYX73/g+4xlS9UOZ6YRpA5s02BTZ6R6r6ARj8HmzjDnJ/MYKEBTcP1qSuNDX3mSrqg9dbgiG6QjlIJff91lM+Z/17olLUNTSu0E/qmyCVfz+mHg/e1uwkkZmhBz06o8KxYKxsBXgl0q73BuIo9lKC7rj9eVcQHT7lFyfW7cvYEvPnYWXwc52is2JvnAj17Ohx98iq6ojVJTFVgnh9KUPM2Vm5PVzb6i6xOyDHZ0R8mUgjx0LGzSG5+a+daq5G0NkTI0IdarVmVmAyNZ9vTHq9clozagGUwFVQy1ZVwHDh2ru0fEMhhOF/n7R88CcFl/nE/+3HXs2RikEbZ/89S0yonODpuxbLDRlohYWKYilQ9mYTi+pitq43o+Q+mgAmJDLHxRz4BrtJRlaPcD+4FepdRZ4LcINuU+BvQB/1sp9T2t9Y8u1TMIsV61quMFpk0vtkyD63Z0T1tt1t7D9YMStXQhqNN91a4ePnbgWvqTU6vWxmPCecfDNk3es38HRwfGqnW6tqFwfF19hkoZWrboYRnOmqvlXYglC8Ba6wMtvvWlpfpMIUSgVR3vrg0d5By/Lki2Wm1W7mEblYnFQTnYhliIT79zL/FIfYCf6VDEnTXX1c6Lq+iNh5nMO3zzrtct2j+DtUBSEEJchJqtRh1P8+E3XQW0d3Ls4L7d/PrnH2eyfDoNIGobfORt10wLvhWzHYo4fHyIVN5hcLJA2ApG0iej9prq37CYJAALsQbN1kVstiO67fx6//RLKVI1wddSQU2u3ebooGbPfPeDTxELm+RLHiXP5/xknqLrEbLMdZHzbSTd0IRYY+baqWw+dU5ffvwcf/Svz6IJKh22d0e5YnOSzo4Q9x0ZmNdzVzYGe+MRtnZHCZkGvtbkSt5qaJ6zImQFLMQaM1sjHZh5dlrlHs1Wz1pr/vzfTvKH//Isvg4OTuzoiRILBymHdnvtNluh127qJSJ29dDHZN5Zl8EXJAALseY0VjikCw5DqQKnR3McOHSsrl+D62lOTWarp8o+9KXvY5hm08D86st6+e1/eprPHXsBCILthnioGnyhvV67rYJ/PGROq8BYr7nfCgnAQqwxtRUO6YLD+YkCGk3EMqrBLlt0iIctzk8WMFCYSuH7mrOTRTZ3humMRuoORrz3bx/lso1JvndmAoAbd/dw4Ie388dfO9FWxUStVit0pRSO114FxnohOWAh1pja6RNDqSD4QlDKVRmGGTTaKWIQ9F5QSlVPqE3mnIYuaJpMya8G37f80BY+884f5i3XbmvZkGcmjYMzIVhNZ4ruvO53MZMVsBBrzP4r+rnt7ASf+tYpcuVmOr2xUPlUWxDsQpZBtuQFfXkJ+jRoDWHLoOj5jGSC4KzRODWterd0Rfij215RrXSYT6/dmXoJr+XevUtBVsBCrDGHjw/xwGPn6EuEiYVMTEMxkXdJFxwgCHZ7+hNc3h/HUArP11iGYktXhO4OG8swKLo+nvbrupkpYCRd5NvPjy7o+WpX6I29hEU9WQELsQzmOv13pveOZ4vVHGtvPOgyptEMpQqYhqoLdo3NdEJWcDT4E4dPUnCbF6jd/eBT3HZ2gqMDY/N63rU+Jmg5yVBOIZbYQgZHNnvv6dEs27qiJKMhYGqkUMH1uWFnT12wqwTvSiC84zW7eOZCmj986Nmm9cGGCiZa5B2fbd3R1Tboci2RbmhCrAbt1O3O9N6S6zGacSl5PiHTwAAupIvVAJyM2limatq+sZJzPXx8iE8efp733v94dcRP2AzywRAEXlMpDKVIFVyUYl7PK+ZGArAQS2y2CcMzee5CilTBrZaSuZ7GB9w2G+pAsAr+0Jd/wFjOqY6LD1sGHbbCdA08DUa5QkLrYAxQh1W/PdTu84q5kU04IZZYZQBmrXYPIFRG+VRKyQwjWKVGQ2bb5Vwfe/gEw5lSNfh2RW22dIYpuuD6mqLrU3A9XM/D0xqlgj6+83leMTeyAhZikTVumt20u4cHHjvX1oq18b0AaPDLgbEyZLgjZLY1LeLEhTTfOztZnTLclwjTHw+RKboUXA9DKSwDPB8cDQpNxDYYywYVFeutQfpykwAsxCJqdgz3gcfOcdt1W+uakjerCmj23qLrk4hYFF2/mgNOxmx2boi3eIIpR0+O8O7PPVYNvls7I/TEwwBcSBUJWQabkhGG00Xyjlee6aa4tC/OSKbIWNbB8TR7+hNSxbBEJAALsYhabbgdHRibdcXa7L3dHTbjOWdaRcJsq9EvP36Ou774fYquT4dtEgubREImWuvgHr7Ptq5otSnOwHAGx/ODIZtK0ZeIEAtba30u26onAViIRbSQDbfKeytlZZUVb8iE/kSk7ZraT3zjef7wX4Iysy2dET71zr0MpYp15WiVsUAVJc9HASFzaltINt6WngRgIRbRTMdw23nvqZEMo9lSteqh5PkYSnHT7h6ODgRButKPtzEIe57P3Q8+xd/8x4sAXLU5yafeeT1bujq4akv99ZV0RyUvbRqqOp5+rs8t5k+qIIRYRAs5hntw327Gc8HmlzKCHg4KRSxk8vHDJ2dswJ4tutz+V49Ug+9r9vTy+TtuZEvXVAA9fHyIA4eOcfO9D3PfkQFuu25rtZJiZ08H3R02pqHk+PAykhWwEItovsdwK9UPjufXNc7pS4QZShXwaqYINx6MuJAq8K7Pfpcnz6UA+Onrt/H7b3s5dk0tb6vNwdrytcZTc7LxtvQkAAuxyOba8avxuHGQj1X0JcIkIjZnxnNErOntHc+O5zhxIc2BvzjGSKYEwLbuKD9+zaa64AvtncaTTmXLT1IQQqyw2uDYWy4TqzTXyZVcLMMgEalfK+Udj0TY5q0f/3Y1+G7tihC2DH7rH5+eNh+uVY9e2WRbWbICFmKZNR62eO5Cis2dUYBqT99Kc53+RIRXbE3yz09eYCRTImwFwdhxNZOFHK6vMRTs6OkgUR4V36xvw0I2B8XSkRWwEIukdpPrwKFjTacUN5tonCl6jGSK1WuSUZtNnZFqZ7NHX5ykJ2YTMhUF12M4XWQ0V8L1Naah2NUbqwZfaL6ylR69q5OsgIVYBDNNIa5diTbLxfbEbMayDrGwNe2wReX6zmiEno4Q5ycL1UqJq7ckCVsGk3mn7lmarWxrp2hkSx6xkMntN++qdkqbb69isTCyAhZiEdQGVqVUdTZbpWa3olkudkMsTCJiNW2uU7ne831eGMtXg2/EMvjCwZu483V72lrZ1k7RuHJTgr5EmAceO8dHv/7ctBV5Y4mbWDqyAhZiEbR7Aq5VLnZPf2Lakd/Dx4dI5R3Ojeeh0ogHSIRNrtqcJBa2ppW9xcMWtqH50FeeZPuRqdVsqyqIT33rFH2JsPT+XSGyAhZiEbTbcrLdXGxtSqMyVBOCxukR2+Td+y+rXrv/in7uv+NGfuct15AteTi+nraabVUFkS15Uh2xgiQAC9FCO5tqFY2BdSRT4Ox4nucupOreu/+K/rZGsweHMrxqyqFWqyFiM6VBWv2AiIXMefcqFgsnAViIJppVK8yUG60NrIOpAmNZh+4Om82d0WnvraxYv3nX67j/jhub/qp/fDDF4GSRSr+ckKmIWAamoeiM2tNyyzBzrW+rlfftN++S6ogVJAFYiCba3VSrVQmse/oTbOuO0peItP3eCq01H3v4BOM5p7rSNQkmYxRcH8fTnBnNcmIoPe29M6VBWq2873zD5W2tyMXSkE04IZpYjLaS7b63Ugb2wmgGx4Phck2wbSp8T+M1XF/yYSJX4vDxoboyshNDadIFl+4Om9749EkWrY4ayxHklSMrYCGaWMgct7m8t5LqeGkyR6rgVoPvK7d18rG3X0s4NJVSUEzNOjdUsKKuTZVsSkboiQUN3Acn87KaXQMkAAvRxELbSjqeZjhdYGA4wzMvpTg7nuem3T3Trr3vyAAKzeBkkUwxCNrJiEXEMrjl5ZvpiYUwVFD9AKAU2EZQGXG23Bu4NlXSG4+wrTvKno3JlvllsXpICkKIJubbVrLy3tvOTvDxwydxfZ+wadDZYfPAY+cAODowVj119vT5CXIlvzqdYmMyTF88zPnJAhCspkfSRTRTo+N9X6MM2NbdsaBUiVh5sgIWYhatyr5mcnRgjG3dUa7a3Mml/Ql64xEcz6trrH56NMNkIajbVcD27ij9iUhduuLgvt0kIhaer/F8P/hLa+Jhi4P7dlfTHemCw8BwhuODKZ4fyhAPy9pqLZD/lIRoYqbeDsCsvROarUwnc061sfp4tsTg5FQDns2dYTqj9rRUx/4r+vnD217JvQ8dZ2AkC8BlvR188MeurH7mBx54gvGcg6GCHLHj+ZwYSrP3d78mE41XOaX1fH6+t3FjpT4D/AQwpLW+pvxaD/B5YCdwGvgZrfX4bPfau3evfuSRR5bkOYVo5sChY9OODOdKLiHTIFvyqs3TK5UGjZtdP/Y/j3BqJIvra3ytUUrh+ZqQqejqCDGUnqp0iIVMrtzcOe9JFLf8yb9xeiyH52sMpXA8H9NQ2IZic1e06fOJJadmv2RpV8B/CfwZ8Fc1r30Q+D9a648opT5Y/vNdS/gMQsxLq9zqiaEM27qj1cDsepqhdIGDn3uU63Z0V1euw5kijufjVdY35YVOydPV4Bu1TfoTIbZ0dSxo9Hum5HFZXxylFAPDGdAGygCnvNqW3g6r15IFYK31EaXUzoaX3wLsL//7NYCsAAAgAElEQVT7zwKHkQAsVqHt3R2cHs2QyrvV8fDJaPA/l8pps1Te4fxkHgX4WlfTFLGQSWfUJp138H0NujJgcyqfnAhbbIjb+Fot+NRZbYOfkudjKoXWUyPmZVNu9VruTbiNWuuXAMpfW/5IVkrdoZR6RCn1yPDw8LI9oBAAN+3uYShdKo+Fh5LnM5Qu0RezGckUGRjO8OJYDtfT+BrCllk98TYwkg1aSJaHa1YXweWvhoLOqMXmzo5FSQ3UlsyFTANPa7SmOmJeejusXqt2E05rfQg4BEEOeIUfR6wzRwfG6IuHSBemVsCJiIUyDIYnCxhqKqC6vmZDuH51PJot4vrT/2urgMv74zz0vh9ZtGetLZmbLE/K6InZxMOW9HZY5ZY7AF9QSm3WWr+klNoMSNdnsSqdGc/RGw/Tl4hUX9Nac2IoQ38iRCrv4nrBwQnLVGSKLv0Eq81dGzo4Ndr8V34NZEuNh4sDC5lMUXucWMbLrx3LHYAfBN4JfKT89SvL/PlCtKVV43QIJlj0xiN1OeCi61VXm2955Rb+5OsnWt77/GSh2sehop2RRrUBOh4yUUqRLrrTgrX0dlg7liwHrJS6HzgKvEwpdVYp9S6CwPuflFIngP9U/rMQq06zo8iTeQdTKY4PpoNqA2BLZxTDUJiGQX8iwk9du4VPfevUtMMbtX0cPF9z5989Xtfacrbua7U9H0wFzw9nOTGUwVTIGKE1bMkCsNb6gNZ6s9ba1lpv01p/Wms9qrV+vdZ6T/nr2FJ9vhAL0di+0TYUCujssFAEm3LnJ/MUXY/+RIT73nE9f/2uG/jcf7xIquBOu59mKmdsGZAreXVBc6ZevlAfoEcyJUylMA3FSKY0p3aXYnVZtZtwQqy02l/lDxw6Vq2rDVsmw+kiBdcjV/L4yK2vYO/Obn7hM99hOFOa/cYawrZRDZr7r+hvmfKoVC/U1iVXSs0oV2eAlJqtVdILQog21K5QExGb3X1xrtyUpDNqc8XmJD/1yaP8+8nRuveoFmehXA0dIbMuaM420igRtqo56JBpBOVtNbW+Umq2NkkAFqINrXr8dnWEeOvHv82zF9LVlpEVzU75GwosQ5EreXVBc7aRRsOZIqm8Q67k0hsP4WmN52t64yEpNVvDJAAL0YaD+3YzmXc4MZTm+GCKE0NphtMFnh9KM5gqELENPvFz15OMWNX+vbXxWKlgtWqbBqYBxfJ4odqgOdNIo86ozYZYiP5EBF/DZX0x9vTH8TXSeH0NkxywEG1SEBwr1hrX1RTKA4t7YiE+9c7ruW5HD88NpvjTh5+vBuGSq9FAfzxMNFTJHfvEQhb3vPlqIMgv19b+tupDMZl3eOh98+8ZIVYfCcBCtOG+IwMkozYbkxEupIsMlxvqKOCS7iipXFD5cOcbLgfgU986RbbkEbENQpZBPGIRtU1MQ1W7kwFNa3/j5VHxrTbkxMVDArBYkxZyamw+zozn6IxYnB3PM5EPlr6KYJWbKrp1hybufMPl1UBc+6yNJ9MOHDpWLS0Dqp3LVLmlZK7k1rW8lBzvxUcCsFhz5npqbDEC9OZkhB+cn6TgBGVfhgrGxYdsc9aWj61Ops2Uavidt1wjx4nXAQnAYs2pPZQATAuA7QTouTg/kef8ZL4afBVgKkApeuNBx7H51OHOVPsrx4nXB6mCEGvOXE6NNTvWOxdPnZvkbZ/4Nucmgg5oOzd0ELIMTNNgS2eUZHkFO58c7UImL4uLgwRgsea0qsmtPTVWG6BTeYfByQLfOT3GgUPH2u6ZcPjZIX720FEupIpEbZP7fv56Dn/gtdz3juvpT0SwTLWgwNl43FnKydYfSUGINefgvt3c/eBTLTepan+1r3QsAwibalo64qNff65asRALmdx+8y7ufMPl3P+dF/nwl5/E9TWWoYiHTT7zrdPYhjHvkfWt8tIScNevJRvKuZhkKKdoNFPP29oc8OBkgZLno1Bs6YqQiASTh/sTEW7a3VNXs+vroFPZ3ku6+c7pYFasZSi2dIZJRkMtB3C2+7yVZ5ppmKe4aLQ1lFMCsLgoVQL0d06PETYV/ckg+EJwkGIy7zCZd8g7HpZhVF8vun7d3LaemE0yGqretxK85zpEs9WU5fncS6wJbQVgyQGLi1LlWO8NO3vY3BWtBl+YyhdnS161f4Pv+5S8qeAbtQ2K7vTJFfPtOjbbxqFYn2YNwEqp9yqlupfjYYRYbDNVGsRCJr6uBN9guGbF7t4YhqE4N1EgXTlzzPxPpM22cSjWp3ZWwJuA7yqlvqCUukWpVk32hFh6h48PceDQMW6+9+G2KhpmqjS4/eZduJ6m6OlpEyxOj+ZIlgdtDk4WFlwmJiVnopm2csDloPtG4JeAvcAXgE9rrU8u7eMFJAcsYPE3sr5xfIg7/voRHG/qfwOKoFl6pd9uZ9QkVfDoT4QXfCJNhmWuK20tVNsqQ9Naa6XUIDAIuEA38IBS6mta6/8+/2cUon2znYCbi7/5jxe4+ytPVXvqbkpGeH44g/ZBoVAKfDTpgsd1O7oXZaNMSs5Eo3ZywHcqpR4F/l/g28DLtdbvBq4HfmqJn0+IqsXYyNJac+9Xn+E3v/Qknq+5tC/Gl3711bw0mUdrKHo+BcfD9Ty01ji+L2kCsWTaWQH3ArdqrV+ofVFr7SulfmJpHkuI6Vr1ToiFzGk9dZutNEuux3/9whP80/dfAuCHd/bwF79wPd97cYJM0cPXGttQuL4maPugsQyqR5gXmnpYrs5tYu2YdQWstb67MfjWfO+ZxX8kIZprNSp+NFtiKF2oa7zTuDk3mSvxjk99pxp8f+IVm/nbX3kVXR0h7jsyQE/MRqEwDIVtBuk7BWztii5o7HvtOPmZnk+sT1IHLNaMZhUNffEwyag9Y+OdM2NZbv3kv/Od02MAvPtHdvOxA9dilwdanhnPsSEWZktXBKvcMB3AMBTJaGhBzXwWszGQuPhILwixpjRuZN1878NNe+pW8sJPnBnn9s8+ynCmiGkofvvNV/GOG3fWXV9JbSQiNomIzfHBFAqqAbrxnnPRquevHMAQIAFYrHEz9dT9+tMXuPPvHidXbrTz0QPX8vorNwL1edl4yCRVnnIRtU1MFeSBK71+a++5mM8nhKQgxJrW6oDDZX1xDn7uUXIlj75EmL+746a64Fubl3X84CBGyDSYzDvs6o3R1WEvuN3kTM8nlRUCZAUs1rjG1pBbu6J0d4T46/8I9o0v7Yvx2V+6gW09UyvOZvXEAF0dIb76X/YBi3doYr6tK8X6IN3QxEWj6Hi87wvf459/MAjAjbt6uO/nr6ezI1R3XSVvXHuqvtIh7Zt3vW5Zn1lctBbvJJwQq91ErsS7PvsIj74Q9PF98yu38Ec//QpCljntWsnLitVCcsBizXthNMvbPvHv1eD7q/sv5U/f/kNNgy9IXlasHrICFvO2Gk54PfbCOL/yV48wmi1hGYp73nI1//erLpnx2SQvK1YLyQGLeVlIZ7LFCtwPPfkS7/v8E8Fx5LDJnx24jtc2jKWX8T9ihchEDLF05nvCa7GO5v6vb5/iV//mMfJO0Cry83fcyGvLwVVOn4m1QlIQYl7me8KrsQTM8zVDqQIHP/co1+3onnU1rLXmd//paT797dMAXL4xzv/6xR9ma80Gmpw+E2uFBGAxL/OpJDh8fIjHXhzH833Clkk8bDGec4DgIETjyPhGRcfj1z//PR56slxmtruHQz+/l2RDsF1o1zQhloukIMS8zLWSoJJ6UAqM8lHfoXQRjUahCJnGjKmC8WyJA39xrBp83/pDW/jrX76hGnxrRxVN5EpM5p15dU0TYjlJABbz0tiZzDYUsZDJh77yZNNZbZXUw8ZEJJi/pkEDrqfxmeq70CxVcGo4y9s+8W0ee3ECgPe+9jL+5Gd/CLtcZtaYVy55ftBMx1Bz6pomxHKTFISYt0pnstqqg9rVZW0qoZKXVaFgc3gkU6TkBUF4S2e0upIdzRbJFj1uvvdhtnd38LqX9fPJIycZy5YwleKSDR18+XvnePSF8WoKodXR4u5YmIfeF4wSmq1rmhArYUVWwEqpX1dKPamUekop9V9W4hnE4mmn6qB2LHsyarO7L87GZJiwZVSb3oxkCgylS3SETLqiNgPDGX7vq88wli2hAKWCdEJjCqGdUUUyFl6sRsu+AlZKXQP8CnADUAIeUkr9b631ieV+FjE/jXW8z11IsbkzWndNbQA8fHyI8WyR06NZbMNgYzKMZRqELJP37N/B0YExzo7nyBY9+uIh+hIRhjMFLqSL1fuFTPBRjGZLhC2TZNSuDuScbUOw1efL6Tex0lZiBXwlcExrndNau8C/AW9bgecQ89CsjjdT9BjJBMEylXcYGM5wfDDNZN7ho19/jrsffArH12zrioKCsxMFQqbBPW++mjvfcDn333Ej37zrdSSjNhtiIU6PZBmcLNZ9ro/CNBQGqvpZlSA/04Zg5Xlbfb5UQYiVtBI54CeB31NKbQDywI8D0465KaXuAO4A2LFjx7I+oGitWb61J2Yzlg0amo9mS0BwDCgWNvn44ZP0xGw6oxEAktFQkEboCE0Lfls6Izz6wjhek8OZlTFBplKUPB+YWuXWHi0+MZSm5PrVFMhErlT3vDN9vhDLbdlXwOVBnvcCXwMeAp4A3CbXHdJa79Va7+3r61vmpxStNMu3boiFSUQscqVgsrABmIZiJFOi5PqMputXs802v0YzRU4MZZoGXwgCuuMFY+JDpjGt7G3/Ff0c3LebjpBFXyLM5s5gmOZzQxnccsCe6fOFWAkrsgmntf601vo6rfU+YAyQ/O8a0Woza09/gmTUZltXFB/QOlitApR8qiN/KtfXbn6dHE5z6yf/vXwoozkNmAp8H6K2QX8iMi2F0Goz8EKq/geAbL6J1WKlqiD6y193ALcC96/Ec4i5mynfur27gwvpIkZ5vLtSQd4W4EK60PTAxndPjXHbJ4/ywujMK1IFRGyTDTGbRz78Ru6/48ZpKYRmq/ONiTCO79c9byrvMJ4tcvO9DzetWRZiuazUQYwvKqWeBv4ReI/WenyFnkPMQ4dtcHY8z4mhTN1mViU4V/7P1xpDKfrjIbSmeiiicv0/PXGen//0fzCec6qBupUdPR1s6oywZ2Oy5TXNVueWabCnL149MBIyDTTg+FpOxIkVtyIHMbTWr1mJzxVzV1tylghbDGeKdEZt9vTHyTse2dJUwNt/RT+X98c5NZLF8zUh06AvEcY0FJf2J7j/jhur1/73v3+CLzx6tvrnroiFYcBwZnoaIhk2KLoegymHybzDgUPHmvZxOLhvN3c/+BS5klvXhvLDb7qqeu2BQ8coeX7dJmKlnE025cRyk6PIoqXGkrNTI1kmcg6up1seuLjrlivoT0bY0dPBrt4YpqHqUg6e5/PL/+u71eCrCHK7qaJLR8imPx6ishjusE22dUUIWSbjOYeemM2mZKTlqrXxeHSzPHE7hzaEWC5yFFm0NK11pNYYKjhGXDk63Bi8Zpo2kSu6/NrfPc7DzwaBMxYyKTgetmWg/eC+u/vi9CcjdQMyDxw6VnfQYqZVa+V4dCsyD06sJhKARUuNfXVDpoHj+dU6XGgevJoFwaF0gds/+wjfPzsJQFfUYmtXlNOjOVxfoxTT6ntbPQfMf9XaKk0hJ+LESpAUhGipcVOrLxHG10GN71yGWZ64kOZNf/qtavANmYqwZWAYQY5Y62B13ay+t9lzwPxXre2kKYRYLrICFi01rhZNQ9HdERwXnsw7bQ2zPHpyhF/5q0fJFIOzNls6I2g0Q+kSSil642E2xIOTdJX63sZ7LvaqdbY0hRDLRYZyihlVqiBqj/hevjHJTbt7ODowNuN0iS8/fpa7vvgDiq6PoYJSskQkSCWMZApkix6dUbutQF55DpliLNaItoZySgAWs2qcMjySKTKcKdGfCLEhFp42dVhrzSe+cZI/+tdnqyfYdm6I0RGe+oVLa1230SbERaatACwpCDGrxmqIdMHFUJDKu/TGI3VVCTdf1suHH3yS+79zBoArNyeI2iaT+fr63sZ2kYsxpl6ItUY24cSsGmtnS16QUqithojaJi+OZrj9rx+pBt/X7OnlC3fcxJ2v2zNru0iZ1SbWI0lBiKZqV6WpvENHyKQvEbSUHBjOUPKCrmS7++IATOZLjGZK1ZNxP339Nn7/bS/Htoy6+50dzxELmSilSBeDvgyxsElvPFL97FzJpT8RqTs5J8QaIykIMT+NM94832coHfT57Y0HrSeHMyWSUQutNRP5Ei9NFvF8jQLe/MotvDia5bV/fLgupdBsftzgZIF8ySNsmdUNOjmZJtYLSUGIaWpzvpmiSyrv4vmawVSRp19KkSt5vPkVm+iKhjg+mOLseAHP11iG4pdevZPHz0wwki01TSk0towMWwYoGK7pGSwn08R6IQFYTFPJ+aYLDucnChTdqUMQplLEwibfPDHCmbEclTSwoaC7w+boydEZB3Q25pMr4+gLrjenwx1CXAwkBSGmqfRLGE4XUeUm6BAEWcNQTOYciq5fnV5hm4odPcGKdWAky57+eN39orbJiaE0Bw4dYzhdZCRdZFNnhETEJhm1KboeuZLX9uEOIS4WEoDFNJWTZwXXwzIUfjnQWoaB1j55N5hQARCxDS7p6SBkmVQ2dPOOV9fs5vxEnvG8w8TpMUwV9OI9O55na5euTkf+yK2vkKAr1h1JQYhpKv0SYiELzw9WvpahUGgcbyr4KgW7emOErCClkHc8dm3oqCs5G8kUGMs5KIJ7gEIT3HMwVZReDGJdkxWwaGr/Ff189O3XcveDT1FyPYbTRYo1sy3DlkEsZFByfUxb1TU/h6l2lNmihwJCloFCoRTYGFimoicWklIzsa5JABYtVXr7/vG/PstQTZXCtq4ov/OWq1FKtezPUCk5O/i5RwEouj6WUlimgVLBn6XSQax3EoBXuZU+putrzQtjOXwdrGJ/963X8DN7t1e/3+pZKvW+CjANcH1wtAY8KA/rlEoHsd5JAF7FGg8tVGpq76F14JvLvWcK7FprPv/dM+UUhE9n1OZjB36IfZf3t3WPSr3vps4I5ycK2KbG9TSOD2FL8Z79l0reV6x7sgm3ijUeWmg2g20+Dh8f4v0PPMHjZ8a5kCrw+Jlx3v/AE9XDEq7n8+v3P84H/+EHlNwg8dsdtavlaJV7zNTDoVLvm4jYbOmKEDINTCPIHd/3juu58w2XL+jvQYiLgayAV7HFHMVT6yNffYaJnIOpFKZSaB8mcg4f+eozvGr3Bm7/7Hf59slRYGpo5tmJPB944Al+/sZLODowxmMvjqMUbExEUCE1bU5b7ey1RMQmEbGrPR5k5StEQFbAq9hijuKpdWo0Vz1UoZTCMBSGgpPDGW7/q/rgGzIVthVMw5jMO3z88EmG0gU838f3Necn86TKrSZrfzgc3Le7ZQc0IURAAvAqtpxBzPc1rg/ffn60+pomODThVYdmBv++I2QRtoKOZgaKkUxQIVH7w0FmrwkxO0lBrGIzjXifi8bNsr54iMFUEaWDwOp5Pm75dIUiOCRROWbs66Dvr6mD7nrhcnvJvkSY8xMFQFPymv9wkNlrQsxMAvAqt9Ag1qySouj6dNgGRden5Goqe2u2qdjR3cFIpsBkoT714fka04BEJPivTLC5BoOTBTQ0HaYphJiZBOAltpR1vO3cu3GcUOWrHQtR9HxOjQQ521jI5JPvuJ7f+NIPcH2wFNVVccXmRBjDNOumJPcnJbUgxHxJDngJLeW4nXbvfWIozUsTeY4PphgYzpAuOIRMxbmJfDX4hi2DWNjkk4dPEg+Zwak10yBqm0Rtk7Bl0GEbKMOQvK4Qi0hWwEuo2eqztlRrqe99+PgQ6YKLrzWmoXB9zdmxHJZlUHCCxINtKjYmwyQjQRAPKho0ng7Kz7QO/uqMBePjJa8rxOKRALyElqqOt91733dkgO4Om9FsCe2D1kEPX7ccfLuiNp0dFslICJhKT0Rsk6F0EcfzCZsGnTEb2zRbVl+s9HFpIdYqSUEsoaWq42333mfGc/TGw2zpjGIoTcmfaiV54IbtdIQM0MGQzUqKwi2PuLjvHddzw84NbIiH2bkh3jLVIFONhZg/WQEvoUpj88qmVaVl43zqeBtXmTft7uGBx87NeO/t3R1cSOUxys1wqq93RXnDFRv5yuPnyTlBu0jbDFIU5yYK7OmPt51qWMo0ixAXOwnAS2ix6ng/+vXn+Pjhk3i+JmwZuJ7PiaF0OeVQAGDXhg4+/Kar6u79Kzfv4gNf/D7juRK+Dmp8u2M2t12/ld/+p6dxvGAFrQkOWZgqyP1Wxgc1e9bGHwTPXUixuTNad41MNRaiPRKAl9hi1PF+/PBJfB1MHXY9zVC6iAIypsGe/jh5xyPn+HXvczyfF8ZzTOQdfB0crri8P8Fdt7yMQ988hW0qUArbCIKur6luvCmlmnZea1ZTnCl6jGSK9CUi1c+WqcZCtEdywKvcfUcGcH0fs6Zvgy4Hy4Lrc2oki+frui5p+ZLLH/3Ls/z2Pz6N52t29HTwlffczEPv28drr9xY7VQWMg0MZRC2TIzgoBuWYRAyjaad15p1Z+uJ2YznHOn5IMQ8SABe5c6M5wibBuV5l7ieX91IMxS4vub8RAHX8zk7nmMiV+KuL/6gGjhfvrWTz93+Kl6+rbN6z8oGXm88jI/G93U1ReGjq6PiG1MJjSPlATbEwiQiltQGCzEPkoJY5bZ3d+D5PqMZBx+Np6eOp1mGgaEUPpoLqSJXbk7ynz/3KMcGxgB43RX9/PFPv5LuWKjunpXNQdtUbOmMcCFdBC/YiNvcGSVZLm9rTCXUtpisyDsee/oTMttNiHmQFfAqd3DfbmzTZEPcxlRUR8RD0CSn6Ho4rk/R9RlKF6rB9+detYM/O3DttOAL9Z3KfA3Xbu/mv75hD5s6o1imaplKkBaTQiyuFQnASqn3KaWeUko9qZS6XykVmf1d60+l4iBXcskWvSAP3HCNr8EnGBF/bqKAoeD9b7yc3/rJq+kIz/4LTiWev2Jb16zHjKXFpBCLS2mtZ79qMT9Qqa3At4CrtNZ5pdQXgH/WWv9lq/fs3btXP/LII8v1iKtCbcVB1DYZzRYZSpdQBHlfRRA8zZrWkRHb4J63XM2t127DMlv/bG28d6WGWIKpEIumca3U1EqlICwgqpSygA7g/Ao9x6rVWHGQyrtBn15fYxtT/9lWgq+h4M9/7npuu277jMG32b0Xa9acEGJulj0Aa63PAX8EvAi8BExqrf+18Tql1B1KqUeUUo8MDw8v92OuuMaKg5LnV0vF0BqjJghbhuKqzUl+5GV9da+3e2+QwxNCrIRlD8BKqW7gLcAuYAsQU0q9o/E6rfUhrfVerfXevr6+5X7MFdfY6yFkGvh6qk+vV96NU0BfPMz73/gylGrrt54l7VEhhGjfSqQg3gCc0loPa60d4B+A/2sFnmNVa6w4SERMXC840labtb+0L8Yf3PryOeVua++dypc4cSHN6dEs49miNNERYhmtRB3wi8CNSqkOIA+8HrhodtgWqzVjbR+JM2NZeuMRskWfdNEFwDYUB/ft4v23XDnve9/70HFOj+awTcW2riiOr6cdPxZCLJ1lr4IAUEr9NvCzgAs8DtyutS62un6tVEEstLqgWfB+9Z5ejp4c5dfuf5zJ8vj3/kSYWNjE8+G267ZydGCsZcCf6QfCgUPHph2syJVc+hMROVghxMK0lQ9ckQA8V2slAC8koDUL3iXX58eu2cTffOdFssWgbeSWriimgtFsiXzJQwObOsOETIMLqSKO77OnL84HfyxYGc/0A+Hmex+mK2rX5Y611kzmHb551+uW4h+REOtFWwFYjiIvooVMwGjsqxu2TMazJf7im6fQBGVm27qCto8vpQoYKHR5ZNBwugSAaShMpTg9luPuB58iFjJn7NXb6mixbMYJsTzkKPIiWkh1QW1pmON5XEgVGMs5aGDnhg6u3tyJZSpGsyUMprqiKcDTGs/XGEphKFXtjjYwkp2x3EyOFguxsiQAL6KFBLTt3R3kSi4l1+P8RIHRbLCqjYctPvOLP8x/e+PluD4UXR+UxtfBaTjTUKCnjhRrHZSsVQLvTD8Q5GixECtLUhCLaCETMH7l5l38xld+wMREgUJ5flDUNvjdt1zN7r44u/vi3APc+XePky25RCyDzniI8bxTDb6+r8vtJCPkHY9dGzrIOf6MY4tkyrEQK0cC8CJrFtBmK01zPJ/uWAiFqgbfjckwv/Wmq/jxV26pu/dH334tH3jgCdIFl7Fc0BvCDBbBKAO2JCJYpsLxNB9+01XAwkciCSGWhgTgJdZsjM/dDz7FbWcnODowxotjWRJhm5dSBSbzTrmb2cv4xVfvrNscq6UByqODTEPRGbb4hRsv4ejAGGfHc/QnInWBVgKuEKuTBOAl1mxq8EimwMcPn2RLZ9CF89kLaTQQsgx+/63X8JM/tIWwZba8X2fUrhuEmSu5HB0Yk9pdIdYY2YRbYs0a30zmHFzPJ1/yODdRqLaV3LWhg7dcu7Vl8G11P2mkI8TaJAF4iTUvTfNRSjGYDg7/hUyDHRs6yBZd7FlaSUojHSEuHhKAl1ilNG04XeDkUJqnzk2iCZqqA3TYJpds6EAB23tibd9PaneFWPskAC+x/Vf0c9t1WxnLlsg7ft0BxbCl2N4dwdca16etICq1u0JcPGQTrsZs5WLz6XR2+PgQf/HNAZzy6IrKUM2OkIEB5Bx/zuVhUrsrxMVBAnBZq3KxSmvG2b7f6p4f+sqTZIteXQ/frqjF5s4omaIrTW+EWMckBVE225y0+cxR+/g3nqfoePg1r1kGlFyfkufLxpkQ65ysgMtm62Q22/cb0xMHbtjOsxfSpApu9XpLBV3Nip4/542zxWr0LoRYPWQFXDZbeddM36+kJ4bSBTojFoOKAfwAAA/HSURBVOfGs/zGl35QDb4hy2BTIkzIMvC1Ihay5rRxVnv/2vSHjA8SYm2TAFw2W3nXTN+vpCeitonjaSbyLpliEKxDlsHGRIgN8RCbu6Js6Yry0bdfO6fVq4yRF+LitK5SEDP9Gj9bJ7OZvv+hrzxJMmJRcHzOT+TJlVfKQTeza/j7R89ybiLPtu4Obtrdw31HBvjQV56c9gytnm8hjd6FEKvXuhlJtNB5bTP5mT8/ytnxLCMZh5IXbLklIxZ7+uN88Vdf3dYzQOvxQfcdGZDZbUKsLW2NJFo3KYh2fo0/fHyIA4eOcfO9D3Pg0LG2cqypgsONu3sYTBWrwbenwyYZsfm11+1p+xlm+p6cfhPi4rRuUhDtVDHMtc53NFPka09f4C++eQpfBxUOnVGby/oT/Or+S6cd4njsxXE83ydsmfQlwiQidvUZNLR8voU0ehdCrF7rJgDPNoCyWdvI2gGWtXxfM5Qu8sCjZ/j/vvYcvg5Gxd/7Uy/npkt7iTR0K6sEd6XAUArX15yfKLClKxgpVHmGmZ5PTr8JcfFZNwH44L7d3P3gUy3H87Rb5/viWJb+eJjuWIiHnx0GYHNnhA2xEB/68pPs6IlNW51WgvvGRITzk3mUBtAMThboT0aqzzDT8wkhLj7rJgDP9mv8TCvkygrWMoLuZc8OZciVgkqHyzfGSeUdciWX7o5Q09RFJbirUJCXH8kUKXkaDXWbgJJmEGJ9WTcBGGb+NX6mFfJ9RwYwjWAE0NmJQvVAxoZYiFjIxPV8YuFg9dwsdVEb3JNRm2TUrlYx1D6PpBmEWF/WTRXEbGZq8/jCaBatNWfG8jXB1yZkKobSRTxfMzCc4fhgioHhDK7n19XoShWDEKKZdbUCnk2zFehIpkg8bPH8cKbaSnJTMkzENtncGWU8W+T54SymUphK4XqacxMFLuuL1d1X0gtCiEbrKgDPpaFNpdLhG89e4PRoDl+DUrClM4JtGvg6WNne+9BxADzt4/tBv18FZIpu3f0kvSCEaLRuUhBzaWjjej7nJ/N84ZEX+c0vPUnJ81EKIqZBwfHY3BmtpifSRZfuDguvJviaBryUKkizHCHEjNbNCrhVne9HvvpM3ar4l1+9kz0bExw6cpK//c4ZAGxTsa0rCgq88uig2g22x8+MEzINDCOocvC1RpU/U1a9QohW1k0Ablbn63o+p0fz7PQ1XVGbwVSeD3/lSTYmIzxxdhKAiGWwvbuDkG1gKDWtwuGm3T0cHRgFwPCD03CGMtjUGZZmOUKIGa2bFESzfr4X0kVsw6AjZAVHiVGM55xq8I3aBtu7o4TLwTd4rf5wxgOPnSNkVla+wQq5K2phmYZMvPj/27v3GLnKMo7j32cuu7Pb7q0tu22hpa0UaK0RSr1AvKCogRouGjRAiBqJiijG+I8o3qIi0USNCl74g3gLihLFxsAfykUIys0WaUHAIpRur1vYW3dnZufy+Mc5u26X7XZ2OzNnZ+b3STaZ7ZyeeZ7O7NOz73ne9xWRGTVMAR5vBdvTP8qOPYNs3zNIJlekOREjXyiSHsuzu3+UbD5YUOdjb13N6UvbKLhj9v+Fjaabvry8s4WmeIxE+K958PAYvf1pzl6zqOp5ikjtaJgCfO7p3Zy1soNXRnM4QUcDwGAmz96BNC/1pxkLdy7ubmtm664Bdr08wp6BDIcOZyb6d4fSOfpHsrzl2/ey9aV+8oUibakknS3BjbjxxT0XLUhyx9Y9uhEnIkfVMAUY4J5n+ibW222Ox0hMKsKFomMGC5vj5ItFBtJjLO9spas1ySsjOfYPZWiKx3AgF44Zm8GegQzDmRwjYwWa4jGaEzEWNMVZsjClXStEZEZ1fxNucu/vUCZPMgbuwToMmEG4IH3cYG13G4VikSJBl8RQOsdwWJzH8kXcnY6W5EQnRU9bij0DafYPZoKhCgA3TmhrBrRrhYjMrK6vgKf2/sYMcsWg+yFfKJIPp7bFDH539dnccc05pPNFWpJxhtI59g6myReceAxGxvL8J5xmPK69JcmJnSmcYJnJmBnLO1O0pYJui8njxSIiU1W9AJvZaWb2xKSvITP7bCVea+ouE12tQWHMOxQm7cR0xRtXsH5ZBwubExPdEocOZ4lhYW+vkUrEScZiHBjOHvEaiXiMjSu7+NmVZ9HdniIeM633ICIlqXoBdvdn3f0Mdz8DOAsYBf5Yidfa3T9KSzKOu5MrFOlsaZroVABIxIyPnH0yX77wtbQ0BYuoj3dLZPNFMKfojjuc0NZMT3vzURfVmWkxHxGR6UQ9Bnwe8Ly776rEyVd0tXJgKE1TIk42V6B3IE3YZcbmDUvZO5DmL/8+wLMHDk8U0fGFcz7z222MjOVJJWIT2weNjuU5tXshna1NR905WQVXREoV6a7IZnYrsNXdb5rpuLnuivzXp/fzlT89Rb7ovHw4OzHs0JGKM5wtkIwZSztSJOKxV+2QXMldlEWk7s3vXZHNrAm4CPj9UZ7/uJk9bmaP9/X1zfr8mVyBtT1tvGtdD4fC4hszaGuOk8kXiRtgxr7BYD3fqS1jGlIQkUqLcgjiAoKr3wPTPenutwC3QHAFPJsTH87m6RvOcue2Xn79yC6KHuzbFjejbzhDthC0jCUt6ETrG86yesmCV7WMaUhBRCopyja0y4HflPukA6Nj7B9M85P7d/KDe3ZSdDhtaRsfPWcVewfTYctYMGMtVwh6e8cKRbWMiUjVRXIFbGatwLuBT5TzvH3DWV4+nOXGu5/hb88FwxbnvGYxX3rver66ZQdNiRheDCZdFMPJGGMFJ5WITdsyNpsF3EVEZiuSAuzuo8Dicp1vfPeKfYNpvnznDnbsHQLgkjOWc+0717Lz4DBP9g6GV7wQjxnJGOTDtRtWL1nA588//YjiOvkm3OQF3CfvdiwicjyibkM7bvlCkf1DGV44NMIX/rCd3v40Blz99jVc9oaVPHdgmBvuegYj6PstFp1C0YnFjOaksWpRK3d/9m2vOu/RFnDXIusiUi41PRU5my+wdyDDv3YPcO1t2+jtT5OMG1+5cD1XvOlklne1cOtDL5KMB+1mYMTjMZJxIxk3uttSXHfBumnPPT6JYzKt7SAi5VSzV8CjY3kODmV54Lk+brjr32TzRdpTCb55yQY2rVpETzgteHwnDDNjeWcwTpzNFzG3GdvKVnS1cnA4M3EFDFrbQUTKqyavgAfTOfYPZrjjn7v56panyOaLLOtI8aPLz+RNaxYHLWfh/myTd8JoSyVZc8JCTl68gI0ru2YcShifkjzdtGMRkXKouQJ86HCWvuEMP75/Jzfd9zwOrFvWxs1XnMn65R30tKeO2MFiroVUEzFEpNIinYpcqk2bNvmjjz7GweEs/SNZvnX3Mzz4n0MAvOWUJXxx8+ks72yhs7Vp2r8/3k423foNIiIVUNJU5JoYA3Zg72Cag0MZvnTnUzy9L2gze//GE7nm3FPoaW+eWIN3OprRJiLzUU0U4Fy+yAt9I1z3h+3sGQjazK55x2v4wFkr6G5vPuJGmYhIraiJypXOFfjUbVsZyuRpSsS4fvM6zj3tBHraU6SmtIqJiNSKmijAu18ZZWkmT0dLkhsu2cDrV3SytCNFMl5z9xBFRCbURAF24KSuFm583+tY072Qpe3/bzMTEalVNVGAU8k4P7rsTJZ1ttDT3nxEm5mISK2qiQK8oquFkxa1Tmz3LiJSD2piEDUZj6n4ikjdqYkCrPFeEalHNVGARUTqkQqwiEhEVIBFRCKiAiwiEhEVYBGRiKgAi4hERAVYRCQiKsAiIhFRARYRiYgKsIhIRFSARUQiogIsIhIRFWARkYjUxLb0ZtYH7DqOUywBDpUpnPmkXvOC+s1NedWWueZ1yN3PP9ZBNVGAj5eZPe7um6KOo9zqNS+o39yUV22pdF4aghARiYgKsIhIRBqlAN8SdQAVUq95Qf3mprxqS0XzaogxYBGR+ahRroBFROYdFWARkYjUVQE2s/PN7Fkz22lm103zfLOZ3R4+/4iZrap+lLNXQl6fM7OnzexJM7vHzE6OIs7ZOlZek4671MzczGqmzamU3Mzsg+H79pSZ3VbtGOeihM/iSjO7z8y2hZ/HzVHEORtmdquZHTSzHUd53szsh2HOT5rZxrK9uLvXxRcQB54H1gBNwL+A9VOOuQb4afj4MuD2qOMuU17vAFrDx5+sl7zC49qAB4CHgU1Rx13G92wtsA3oCr/vjjruMuV1C/DJ8PF64MWo4y4hr7cBG4EdR3l+M3A3YMCbgUfK9dr1dAX8RmCnu//X3ceA3wIXTznmYuAX4eM7gPPMzKoY41wcMy93v8/dR8NvHwZOqnKMc1HK+wXwDeA7QKaawR2nUnL7GHCzu/cDuPvBKsc4F6Xk5UB7+LgD2FvF+ObE3R8AXpnhkIuBX3rgYaDTzJaV47XrqQCfCOye9H1v+GfTHuPueWAQWFyV6OaulLwmu4rgf+v57ph5mdmZwAp3/3M1AyuDUt6zU4FTzewhM3vYzI45bXUeKCWvrwFXmlkvcBdwbXVCq6jZ/gyWLFGOk8wT013JTu2xK+WY+abkmM3sSmAT8PaKRlQeM+ZlZjHg+8BHqhVQGZXyniUIhiHOJfiN5UEz2+DuAxWO7XiUktflwM/d/btmdjbwqzCvYuXDq5iK1Y16ugLuBVZM+v4kXv3rz8QxZpYg+BVppl895oNS8sLM3gVcD1zk7tkqxXY8jpVXG7ABuN/MXiQYe9tSIzfiSv0s/sndc+7+AvAsQUGez0rJ6yrgdwDu/g8gRbCgTS0r6WdwLuqpAD8GrDWz1WbWRHCTbcuUY7YAHw4fXwrc6+Eo+zx2zLzCX9V/RlB8a2EsEY6Rl7sPuvsSd1/l7qsIxrYvcvfHowl3Vkr5LN5JcPMUM1tCMCTx36pGOXul5PUScB6Ama0jKMB9VY2y/LYAHwq7Id4MDLr7vrKcOeo7kGW+m7kZeI7gTu314Z99neAHF4IPw++BncCjwJqoYy5TXn8FDgBPhF9boo65HHlNOfZ+aqQLosT3zIDvAU8D24HLoo65THmtBx4i6JB4AnhP1DGXkNNvgH1AjuBq9yrgauDqSe/VzWHO28v5OdRUZBGRiNTTEISISE1RARYRiYgKsIhIRFSARUQiogIsIhIRFWARkYioAIuIREQFWBqOmb0hXNc1ZWYLwvV4N0QdlzQeTcSQhmRm3ySYGdkC9Lr7jRGHJA1IBVgaUriWwWME6wyf4+6FiEOSBqQhCGlUi4CFBKuupSKORRqUroClIZnZFoIdHVYDy9z90xGHJA2onhZkFymJmX0IyLv7bWYWB/5uZu9093ujjk0ai66ARUQiojFgEZGIqACLiEREBVhEJCIqwCIiEVEBFhGJiAqwiEhEVIBFRCLyP5NjmxBz8ef6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.lmplot(x='x', y='y', data=df);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们随机生成了一些点，下面将使用PyTorch建立一个线性的模型来对其进行拟合，这就是所说的训练的过程，由于只有一层线性模型，所以我们就直接使用了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "model=Linear(1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中参数(1, 1)代表输入输出的特征(feature)数量都是1. `Linear` 模型的表达式是 $y=w \\cdot x+b$, 其中 $w$ 代表权重, $b$ 代表偏置"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "损失函数我们使用均方损失函数：`MSELoss`，这个后面会详细介绍"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "criterion = MSELoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "优化器我们选择最常见的优化方法 `SGD`，就是每一次迭代计算 `mini-batch` 的梯度，然后对参数进行更新，学习率 0.01 ，优化器本章后面也会进行介绍"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "optim = SGD(model.parameters(), lr = 0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练3000次"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 3000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "准备训练数据: `x_train`, `y_train` 的形状是 (256, 1), 代表 `mini-batch` 大小为256, `feature` 为1. `astype('float32')` 是为了下一步可以直接转换为 `torch.float`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x.reshape(-1, 1).astype('float32')\n",
    "y_train = y.reshape(-1, 1).astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开始训练了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 0, loss 104.3753\n",
      "epoch 100, loss 0.6677\n",
      "epoch 200, loss 0.1083\n",
      "epoch 300, loss 0.0958\n",
      "epoch 400, loss 0.0884\n",
      "epoch 500, loss 0.0826\n",
      "epoch 600, loss 0.0782\n",
      "epoch 700, loss 0.0747\n",
      "epoch 800, loss 0.0721\n",
      "epoch 900, loss 0.0700\n",
      "epoch 1000, loss 0.0684\n",
      "epoch 1100, loss 0.0671\n",
      "epoch 1200, loss 0.0662\n",
      "epoch 1300, loss 0.0654\n",
      "epoch 1400, loss 0.0648\n",
      "epoch 1500, loss 0.0644\n",
      "epoch 1600, loss 0.0640\n",
      "epoch 1700, loss 0.0637\n",
      "epoch 1800, loss 0.0635\n",
      "epoch 1900, loss 0.0634\n",
      "epoch 2000, loss 0.0632\n",
      "epoch 2100, loss 0.0632\n",
      "epoch 2200, loss 0.0631\n",
      "epoch 2300, loss 0.0630\n",
      "epoch 2400, loss 0.0630\n",
      "epoch 2500, loss 0.0629\n",
      "epoch 2600, loss 0.0629\n",
      "epoch 2700, loss 0.0629\n",
      "epoch 2800, loss 0.0629\n",
      "epoch 2900, loss 0.0629\n"
     ]
    }
   ],
   "source": [
    "for i in range(epochs):\n",
    "    # 整理输入和输出的数据，这里输入和输出一定要是torch的Tensor类型\n",
    "    inputs = torch.from_numpy(x_train)\n",
    "    labels = torch.from_numpy(y_train)\n",
    "    #使用模型进行预测\n",
    "    outputs = model(inputs)\n",
    "    #梯度置0，否则会累加\n",
    "    optim.zero_grad()\n",
    "    # 计算损失\n",
    "    loss = criterion(outputs, labels)\n",
    "    # 反向传播\n",
    "    loss.backward()\n",
    "    # 使用优化器默认方法优化\n",
    "    optim.step()\n",
    "    if (i%100==0):\n",
    "        #每 100次打印一下损失函数，看看效果\n",
    "        print('epoch {}, loss {:1.4f}'.format(i,loss.data.item()))       "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练完成了，看一下训练的成果是多少. 用 `model.parameters()` 提取模型参数. $w$, $b$ 是我们所需要训练的模型参数\n",
    "我们期望的数据 $w=5$，$b=7$ 可以做一下对比"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.994358062744141 7.0252156257629395\n"
     ]
    }
   ],
   "source": [
    "[w, b] = model.parameters()\n",
    "print (w.item(),b.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再次可视化一下我们的模型，看看我们训练的数据，如果你不喜欢seaborn，可以直接使用matplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8XFd5wP3fnU2zSTOSZrSOJCuSvG9xLK9ZyYKzkDQhQKG0hCUphdKWNoX2fUspHxpoS15Km0LBIUBYUwg4TiB74iSOHVuSt3iRZVmWJY3WGUmz7zP3/UOZq32zdvt8Px8+rkb3zj0S9Jmj5zznOZIsywiCIAhLn2qhByAIgiDMDhHQBUEQLhMioAuCIFwmREAXBEG4TIiALgiCcJkQAV0QBOEyIQK6IAjCZUIEdEEQhMuECOiCIAiXCc18Psxms8nLli2bz0cKgiAseUeOHHHLsmyf7Lp5DejLli2jrq5uPh8pCIKw5EmS1DKV60TKRRAE4TIhArogCMJlQgR0QRCEy8S85tDHEo/HcTqdRCKRhR7KkqfX63E4HGi12oUeiiAIC2DBA7rT6SQzM5Nly5YhSdJCD2fJkmWZ3t5enE4n5eXlCz0cQRAWwIIH9EgkIoL5LJAkidzcXFwu10IPRRCuaE6vk9qOWlxBF3aTneqiahwWx7w8e1Hk0EUwnx3i9ygIC8vpdbK3YS+heIh8cz6heIi9DXtxep3z8vwFn6ELgiBcLmo7arHqrWRlZAGQlZFFb6iXx2oeoyK7Ys5n7CKgj/Av//IvmM1mHn744TG//8wzz7B8+XJWr149zyMTBGGxcwVd5Jvzla8bext58fyLBONBJEkikojQ4e/gnhX3zElQXxQpl+lwep3sqd/D7rrd7KnfM29/yqQ988wznDlzZl6fKQjC0mA32QnEAgC4Q25eanqJlJyi0FxIPBnnjOsMyVSS2o7aOXn+kgroc5WfeuSRR1ixYgW33HILDQ0NADz++ONUV1ezYcMGPvjBDxIKhTh48CDPPvssf//3f8/GjRtpamoa8zpBEK5M1UXVeCIefFEf53rPEY6HUavUFGUWYdQaMelMdAQ6cAXnpnhhSQX0ofkplaQiKyMLq946o0+7I0eO8NRTT3Hs2DF+97vfUVs78F733XcftbW1nDhxglWrVvHEE0+wY8cO7r77br71rW9x/PhxKioqxrxOEIQrk8Pi4J4V92DUGmnztmHUGEmlUjT1N3HWfZZ4Iq5Uv8yFJZVDH5mfAjDrzHQHui/5Pffv38+9996L0WgE4O677wbg1KlT/NM//RMej4dAIMD73//+Me+f6nWCIFwZHBYHDosDd8jNy00v0xvqRaPSEEvEON59HEemg+qi6jl59pKaoQ/NT6UFYoEZf9qNVe73wAMP8D//8z+cPHmSr371q+PuZJ3qdYIgXDmcXifHOo/h9DlRS2riyTjBeBC1Ss3Gwo1zVuWypAL60PxUSk7hi/rwRDwz+rS7/vrr2bNnD+FwGL/fz3PPPQeA3++nsLCQeDzOL37xC+X6zMxM/H6/8vV41wmCsPRdShFGeq0vkohwTeE1GLQGAvEAK20r+fj6j2PVW+dsvEsqoA/NT3UHujFqjTMu/9m0aRMf+chH2LhxIx/84Ae57rrrAPj617/O1q1bufXWW1m5cqVy/R//8R/zrW99i6uvvpqmpqZxrxMEYWm71CKM9FqfSWei1ddKUk5izbCCBHqNfs7y5wCSLMtz9uYjbd68WR55wEV9fT2rVq2atzFc7sTvUxBmx576PYTiIWWTEIAv6sOoNXLvqnvHvW933W7UKjX7mvfRGejErDMjyzKukIu7qu7iExs/Me1JqCRJR2RZ3jzZdUtqUVQQBGGmptprpcHdgCfqwR/1Y8mwUJVTRY4xZ9IiDLvJzpstb5JvzifXkEtHoANPxEOuMZc8c96c9nWZNKBLkvQj4C6gR5blte+99i3gA0AMaAI+KcuyZ85GKQiCMAvSaRSr3kq+OZ9ALMDehr2jUrdOr5MLngtoJA05hhzCiTCH2w+z2r6aEkvJmO+b/pCQJInm/mauyr6KzIxMStWl5BpyqS6qJikn5/Tnm0oO/SfArhGvvQKslWV5PXAO+MdZHpcgCMKsm+peltqOWtba15KSU4QTYfQaPaF4iOfOPUeDu2HYAunIXLtRa8SkNeGL+OiP9JOhzmBr8Vb02rnNn8MUArosy28BfSNee1mW5cR7Xx4C5qc3pCAIwgy4gi7MOvOw18w686idm66gizJrGVsdW8nQZOD0OekKdJFjyGGVfdWwBdL0h0QsEeOw8zAH2w5i0BgIx8PsKNnBxvytXOwx0h+eWUXeVMxGDv1TwP+N901Jkh4CHgIoLS2dhccJgiBcmvRelqELnWPtZUlfZzPasBltvON8B5PWhFVvVWb2gJJmUUtqajtqSckp+sJ99Ef78YX9/Gy/zNk2DWDm/z67ec77os+obFGSpP8XSADjFmDLsrxbluXNsixvttvn9s8NQRCEiUx1L8vI67r8XSTlJFU5Vco16Zm93WTnpOskKTlFm6+NRCpBzL8JV/NXOduWDcAntpexdVnFnP98lxzQJUn6BAOLpX8iz2ft4xJgNg/8SdfR0cH9998/4bXf+c53pt3Q64033uCuu+665PEJwpVqqntZRl5nN9lZY1+DzWRTrknP7KuLqukOdNMV6CIRKeT4iQdpa7sRAKPBx7//SYyv3bN2Xn6+S0q5SJK0C/gycIMsy1dEe8FkMolarZ7WPUVFRTz99NMTXvOd73yHj3/840ovGUEQ5la618p0rksvfPqiPsw6M4FYAE/Eww1lN+CwONhRfAvfftaGLA+G1F3bDpJtTuGNzt8Zv1MpW/wVcCNgkyTJCXyVgaqWDOCV9/qgHJJl+bNzOM45dfHiRXbt2sXWrVs5duwYy5cv56c//SmrV6/mU5/6FC+//DJ/+Zd/SXV1NZ///OdxuVwYjUYef/xxVq5cSXNzMx/72MdIJBLs2rVr2PveddddnDp1imQyyZe//GVeeuklJEniwQcfRJZlOjo6uOmmm7DZbOzbt4+XX36Zr371q0SjUSoqKvjxj3+M2WzmxRdf5G/+5m+w2Wxs2rRpAX9bgnB5GVmX7sh0cKL7BHWddUiyxKaiTdxeeTv3rLiH2o5aZcZ+Q9kNFGYW88kf17CvoUB5v5uuPkOOtYtgLEihefWcV7YMNWlAl2X5o2O8PCc9Yr/23GnOdPhm9T1XF2Xx1Q+smfS6hoYGnnjiCXbu3MmnPvUpvve97wGg1+t5++23Abj55pv5/ve/T1VVFYcPH+Zzn/scr7/+On/913/NX/zFX/Bnf/ZnfPe73x3z/Xfv3k1zczPHjh1Do9HQ19dHTk4O3/72t9m3bx82mw23282//uu/8uqrr2Iymfj3f/93vv3tb/OlL32JBx98kNdff53Kyko+8pGPzN4vSBCuYCPr0ls8Lfzo6I/I0GRQailFRuZA2wG6A908sPGBYTtEH3/rAo88/7zy9Ye3mPBof0kimUCnzqfMXoZapZ7zypahxE7R95SUlLBz504APv7xj/Pf//3fAErwDAQCHDx4kA996EPKPdFoFIADBw7w29/+FoA//dM/5ctf/vKo93/11Vf57Gc/i0Yz8CvPyckZdc2hQ4c4c+aMMo5YLMb27ds5e/Ys5eXlVFVVKePbvXv3rPzcgrCUjTW7dvqdk+4CTRt5BmhXsIuYHENKSZh0JrwRL33hPvb59uGP+fnCli/Q5jbw0ccPKe+x7aocfv7prWjUKpzeyintQp0riyqgT2UmPVdGttBNf20ymQBIpVJYrVaOHz8+pftHkmV5Stfceuut/OpXvxr2+vHjxye9VxCuJE6vkxfPv8hrza+Rb85nnX0dbd42njr1FDtLdlJmLRt3F+hQI89Y8Ea8qGQViWQCb8RLY18jOpUOnaTD2e/n2m+eGHb/Nz4iY9B38aNjP1QC+ER9Xubakuq2OJdaW1t55513APjVr37FtddeO+z7WVlZlJeX85vf/AYYCL4nTgz8l7tz506eeuopgHFb6N522218//vfJ5EY2I/V1zewV2toO95t27Zx4MABzp8/D0AoFOLcuXNKnr6pqUkZnyBcqdJpklOuUxSaC5Ua8HO958g15NIV7JryiWYjz1iw6C2kpBQatYbOQCcZ6gxk1Fy88GEOHL1Tue7Wze9y/Zb/4/+r+Srvdr07q0dizoQI6O9ZtWoVTz75JOvXr6evr4+/+Iu/GHXNL37xC5544gk2bNjAmjVr2Lt3LwD/9V//xXe/+12qq6vxer1jvv9nPvMZSktLWb9+PRs2bOCXv/wlAA899BC33347N910E3a7nZ/85Cd89KMfZf369Wzbto2zZ8+i1+vZvXs3d955J9deey1lZWVz94sQhEUunSaJJ+MYtAblrM6m/iaseiveyOD/D461C3SokfXmBaYCdJIOnUpHf6Sf9o7NvPvupwmH8wDIL3yDNeu+R0FOiHAiTLY+m3e73521IzFnSrTPZXg1ylK3GH6fgjATk3VD3F23m3xzPofbDxNNRDFqjciyTF1nHVdZr8JqsLLdsR2YWrvbsfLwvz56lt8cHFznKrJ3ksp6gjyTnQxNBittKznaeRSDxoA75ObvdvwdAD3BHmo7atlUsGlWc+iifa4gCEvOVLohptMkVTlVHHYeBiAlpyjOLKY33MtK20pScmpYrfhEhtabN7uD3PToG8BAMNdqorx/20F0mhQvN/lJyUnlw8KoNdIf7ifXkAuAO+TmrZa3yDZkT9jJcS6JgA4sW7bsspidC8JSN7LqZGjPlHRQrC6qVoJ+dXE1J3tO0hPo4abym9iYvxGn3zmsVnxkW9yxZv+hWIJbv/0W7Z6wcu1f3eEj2xyhsU+HN+pljX0NTZ4m4qk4yVQSg8bAhcgFriu9jpSc4kT3CSRJYkPehlH9Xq6ogD6VChBhcqIDg7DUjaw6gYE8+NBDJdLb8ms7agnGgqy1r0W2y8iyjNPvHDfNMdbs/5mzeznTdDXPv9uvXPffH72aet/PcEU9NHX6sOgtbC7aTI4hh3fa3kFGptXbSnFWMR9e/WHicpzuQDfReJTrSq4b1h5g5Njn2oIHdL1eT29vL7m5uSKoz4Asy/T29qLX6xd6KIJwyUZ2Q3QH3ZzoOUEkEWFP/R4lWKf/MzRIp7fkj5fmGDn7P9OaxS/etgIDwfyBHcv46gdW0+5r5/dvNqFRacg2ZBNJRDjsHDjcYqtj67B8fHrGD5ClzyIYDw575lidHOfSggd0h8OB0+nE5Rp/JVqYGr1ej8MhWtMLS1c6nQIQiUfY37YfWZa5vux6pSwwHaydXieP1TyGO+Qm35RPVU6VMjseK82Rnv23ueE/nhss8FNre/ijnee4c20ekiQNHG6Rt5YzrjNEEhGlt/kp1ynuW3Wfct/IGX8kEeFA2wEApQ5+Kjn82bTgAV2r1VJePn/NawRBWLyGplPebnkbq97KhvwN2IyDaYz0jHhvw15cQRcF5gKiySiH2w+ztXgrOcYc6l317KnfMyxXbtTY+bufqUkkBzMBZRVPYMuSCCfyefSdR3l4+8PK4RaZukwa+xrpj/STpc/CmmEd9iExcsZfnj0Qx9r97eg1+jFz+HNtwQO6IAjCUOl0SvqQ5rqOulGHNKeDaUFmgVK6CNDY10hBrIALngsYtUY6A52803aIR57tw90/2ECr4qoXkDLOolFpcGStAMAT9fCV17/ChsINdPo7iSajeKNeLBkWCswFo84SHSvfX2YtQ6/R89Dmh+b4tzQ2EdAFQVh0JjukOR1Mq3KqeO3Ca/hiPuKJODE5Rr4xn5W2lZxxnaG9ayUnmwZ3fRcWHCUr920aXA2sMqxiuW05AA3uBjLUGXjjXowaI8+ee5Y19jU4shx4Ih6a+pt42PHwsDFO9fSj+SR2igqCsOiMdUizSlJxqucU1UXVw7bsS0iE4iHa/e30BHo433ueuoteXjzwAU42DTS00xtaURd+EVteDavtqzFnmOkN9+KL+Khpr6Er2EWbrw2j2kg0GWWNfQ3BeBBv1IvVYGVnyU6c/uFb+qd6+tF8EjN0QRAWHSWPnTGQx/ZEPFj0lmF57L0Ne2nsa0Sv0SPLMjajjYqsa3il5kZ6hrzXxnVP0eI/RZbKSkpOYdaZ2VCwgYOtBzkcO4xWpUWlUhGMBTFYDbT72nFkOfBGveyqHDjfICWnRpUfpvP9L5x/gbdb3kaWZDYXTrqZc06JgC4IwqIz8pBmGNzGD4PB9JG3HqHN14ZBnYnz4oe5ELAo71FW/huuq3RwuL2LaCKKPcuu3F9uLaepr4n+SD/+iB+VpKIyuxKbwUZvuBdPxIPVYFXea6JUSiwZ49qyayctm5wPIqALgrAgJurZMrR8ceSRb2kOi4PbKm/jO68009KxQnl94/IGNMaDnHadZt/FRhKpBJm6TBJygqLMIgDCiTBqlZoySxnLrMto9baiklS0eFvQqXVTbiEwlZ2t80nk0AVBmHfpGu5QPDRm69mRhzSH4iEy1Bk83/g8e+r34PQ6efl0F198UqcE89J8N/fd+DrmrKNIKomdpTu5vux6llmXkSSJQWVALakJxAL0hntRoaLAXEC+OZ8VthWYdCbCiTAalYaHtz9MiaVkwoOkYSA1ZNaZh702WYfHuSRm6IIgzLvJZrZDZ++SJNET6GFZ9jLMOjMX3SG++OTgQROZeolVq/4Pd6SNMz064nKcHH0OO0t3YjPa2FGyg+b+Zs66zxJKhJBkiZ0lOyk0FdIX6SMUD5Gpy6TUUkqOIYdyaznVjmqqmXxxc7FVuoiALgjCnJgopeIKulCr1Jx2ncYb8WLRW6jIriAYC47agbnv4j76w/3YDMU8uldNf3AweL70N9djMnj5yfGjuMPZxBIxGvoa0Kl1w8YyVn34nvo9tHnb6AoMVLgEYgHUkhqr3orT65xSymQqqaH5JFIugiDMuslSKipJxVstbxFNRLHqrUQTUd5qeQtPxMNjNY9xpOMIp12n6Qv3EUvEaW65lm/+zk5/cGCX5yeuT/L/3N/OioJMajtqKc8u5+bym7m96na2Fm3ForfQ2NeojGesWXN1UTVqlZoCcwF6tZ58Uz7ZhmyKM4unfPLQyNTQROmZ+SBm6IIgzLrJUioygx1WJQb+DcVDHO8+jkFjGNjOn4jy2yPtnDp/h/K+N6yS+eBWGX/Mj1E7EKDTO0r9UT+WDAu5+lx6w710+bsmXNRMB+PHah4jISfI1ecq/WB8Ud+UFzaH9lNfaCKgC4IwymSnBk1msja4sixzXcl1HOs+xhnXGWRkkAfO9Mw35XOhJ86hd9+n3KvX93Nr9VFuqbgB/5AAPdaO0sb+RorMRYQSoXH7oqc5LA4qsivYUbIDlaQac6xLiQjogiAMM5VTgyYz1mJhi6eFdn87u+t209TfhFFjJJFMsNq+Gr1Gz9utbxMPaKk9to1USq3cZyv9bwotOkos140K0Hvq97DWPtAZMb2jNBwP0+5v559v+OcpjXexLWzOhAjogiAMMxu11SMXC1s8LRxoO8DOkp1Kq9lfn/415dZybEYboXgYd/uHCAYGO6/mFP6KmOY4CZWG6uL7qMqtGnU26FR2lE53rAu9sDkTkwZ0SZJ+BNwF9MiyvPa91z4E/AuwCtgiy3Ld+O8gCMJSMpVTgyYztA1ud6Cbdn87O0t2Ki1my7PLKc4qJhAPcOS8lfrm7cq9KvPzXFPVQ1yOE4k7KLWUEk6Ex6ztnmxH6aWMdSHa3s6WqczQfwL8D/DTIa+dAu4DfjAHYxIEYQHNNAUxMv9+R9UdPN/4/KgPCZN8NYePr1O+riqQyS76Oa3ei3iimVj1VspsZZh1Zjr9nWwp3jLqWbM1u15MC5szMWnZoizLbwF9I16rl2W5Yc5GJQjCgplJF8HxyhVVkkrpjtgfhC/8WMXrRweD+b9+JMED7/Ni1Wdxe+XtVGRXUGopJVOXSZu3jTZfGw3uBmWXaNpiKxtcaCKHLgjCMDNJQYyXfw/FQ/SGPHzvRTPtfYNh55H7szGaugdm83o7D256kJqOGrIN2XQEOmjsa8Tpc3LbVbexyr5qzAXay2V2PRvmPKBLkvQQ8BBAaWnpXD9OEIRZcKlBcrz8+3N1ag6dy1Reu2ZFI/9w606qHaNn/YWZhdR21CptcYfm3he6+dViN+cBXZbl3cBugM2bN8tz/TxBEObWRDXqI/Pv77bA469rgIFgvr4sxqduVBGM26npqKEws3BUYB76YbK7bveMF2ivJCLlIghXqEvZPDRZjXp6kdLtU/Gfvx9cVDVmJPm7D/jJy5xeKeTlVCM+HyZdFJUk6VfAO8AKSZKckiR9WpKkeyVJcgLbgT9IkvTSXA9UEITZM1mvlfHueazmMY52HuV0z2n6Qn1kZWRh1Vup7agFIFtfwJOvVg0L5j/9zAr+5gNd2MzTbzO7GI95W8wmnaHLsvzRcb61Z5bHIgjCPJnu5qH0B8BFz0USqQSt3laOdBzhtqtuo9JWSZe/m4d/c4Knjwx+IDz20av5wIaBAyV6609e0kz7cqoRnw8i5SIIV6Dpbh6q7aglmUriiXhQS2qy9dn4Y35evvAyDe05vHysGBgI5tetTHLtmi4SuhhOb/WwVEz6Oel68eU5y9lTv2fCtI+oYpk60T5XEK5A6dz0UBPNmF1BF52BTsosZaTkFLFUDCleSsOZv+LlY3kAlNv0fPGeFnZt8lOQOTyNM1a9+JaiLdR01Ewr7SNMTMzQBeEKNN0dlnaTnUPOQxRmFqImixffuR5ZHmygtf9LN1HX/RKhuGXcNM7Imfae+j2L6jzOy4EI6IJwBZpubrq6qJrnG1/g5bpKXP15yuv3be/guuU2SnKMvHBhemmc6aR9ZtrO90ohArogXKYmC4LTyU0/fyLKKwfvUb5ed9VFbl2fRK1SKxUn0y0xnOr1s9HO90ohcuiCcBm6lLLEsbzT1Muyf/gD33j+LACbysw8+qcxPrBJS4mlZFhQnW6J4VSvH1qRo5JUo0olhUFihi4IS8zImbcj04HT7xw2E59pT/NOb5jt33x92Gt1/3QLNnPGuPdMN40z1etno53vlUIEdEFYQoamH9QqNX849weOdR9jY/5Gdjh2KDPxQDTASvtK5T530E1DX4MyQx8vBx1LpLj/+wd51+lVXvvd53awqTR7SuObbonhVK4Xu0WnTgR0QVhC0jPvWDJGbXstPaEe7EY7rpCL2o5athZvxaq30uptVYKgO+jmcPthVJKKEkuJEvRH5qC/+UI9P3jzgvL11/9oLX+6rWzUGOZ7gfJyOlForokcuiAsIa6gC7POTGNfIyadiWQqSaY2k2QqiUlnorGvEbPOTLY+W8lPN/Q1oJJUpOQUy3OXj8pBv3y6i2X/8AclmH9gQxHN37xj3GA+G7n56RA9z6dOzNAFYQlJpx+8ES9WvRWj1kggFsCsM2PQGOiP9BOIBVhhW6Hk0p1eJyWWEpbnLleOaTPrzBxqvsAXn/yD8t5Wo4a3vvQ+svTacZ8/G+eNXgqxW3RqREAXhCXC6XXSG+rltebX8Mf8xBIxsvXZdPo7KckqIRgPolVrlXTE0CAYioeU4BuNw9d+K+MPL1fe+6/u8JGh78MXXUOWfvzAKRYoFzeRchGEJSCd6jBoDdxy1S0UmAo42n2UpJwcSEfoBtIRa+xrRqUj0uWB3oiPn+2Hh3+uwh8emIV/8sYUj30yRVW+eUqlgENbBriDbt5pe4dnG56lqb9JbNlfBCRZnr8zJzZv3izX1dXN2/ME4XLxwyM/5JTrFPFkHIveQlVOFf6on3Z/OxXZFcMWJ8datPz9iV7+7fnBgLt2mZsHb7SikgbndCk5Rb2rnhW2FeMueKY/WJKpJKddp1FLapKpJGvy1qBWqdlStGVUCaVIlcycJElHZFnePNl1YoYuCIuc0+vktebXUKHCqrcSTUQ57DyMSWeiIruChzY/xL2r7lWC+dBFy8buKNd+84QSzFfkZ3L267v49A1Zo5pztXhauOC5MOGCZ3qBst3fTjwZx2qwsq1kG+XZ5SRTSR4/9rhotrWARA5dEBaZkTNsd8hNvjkfSZJQSSqMWiMAJ3tOsta+dlj72d5QL1a9FU8wzld+pSIlD57juf9LN1GSM3DvWKWAp3pOsTZv7aQLng6Lg4rsCnaU7Bg2w+8IdJBIJkSzrQUkArogLCJj9S3Z17yPqwuupqm/CYB4Mk6zp5k2bxs9/h62lWyjzFpGIBbg1Quv09F6N60uq/KeO9YfBV09knoDMBDQx9qlWZFdQZl1eKnieAueY232EQumC08EdEFYRMYqC8wz59EeaGdr8VaOdB3h3a53MWqNFJgKsBqsnHGdIVOXyfELNg4d+RPlvTZUtrH+qnZC8RBJOW/MmfbIdrZT3ZE51gxfLakpNBcOu07s6JxfIqALwhy41N2UY81y1+Wt49ULr6Ir0pGpy2Rt3lpScopQPES2IZs2l56v/Xqwpa1Wf4E7t1zElKEnFI8QjAWpLq4edX7nWD1hajpqgMl3ZI41w39w04PUdNTgi/rEjs4FIgK6IMyymbR7HSuVodfoubn8ZoxaI23eNhwWBytyVnCis4VfvDK8M+E/3ufl7bYaznv6CUQD5Bpz2eHYgV6jV3LvY42xxdPCC+dfIEefQ6u3lRx9DsttyydtrjXye4WZheL8zwUkArogzLKZ7KYcr2/J0A8DXyTED1+10NY7OCu/pfo4N1Uto7m/maLMIlQqFctzliMjc9J1knAizAMbHxhzjO6QmzOuM2gkDUiwPm+90sZ2usFY7OhcWCKgC8Ism+7i4MjUR7qWe6xZbm1DMb88NJg6uXmDi7D2ZaKo2d/ahgoVK20rWWlfSWNfo9IioMBcMCzQDh1jui9MunWAqE5ZukRAF4RZNp12r2OlZ2o6akalZ1463cWf/+yI8vXVy5LsuqYLlUqiJ7CGZdnLMOvM7D27l9Ou02wr3sZ2x3ZgYMPQyA+ToWNMB/1wIowlwwKI6pSlSgR0QZhl02n3Oiz1EXTT2NdId7CbVm8rX9jyBVp7o3xs91nleqtRw/4vvY/M9xpSTD24AAAgAElEQVRo7anfg1FrVD48CjIL8IQ9NPY1YjMNNOIa68Nk6BizMrLoD/eTklOsLV477j3C4id2igrCLJtOu9d0O9x0z/JoMkq+KZ9OXx/v/8/aYcH8r+/08eCu83ij3aPuT6vKqSKZStId7J7wWLehY7RmWEmkEqy2rybHmDPp0XHC4iVm6IIwB6a6OJhOfQzmsY3sP1lGS9dO5ZpP3phiUzmAGV80NSy3PTK9YzPaWJO3hnZ/+6SVJkPHmM7ji+qUpW3SgC5J0o+Au4AeWZbXvvdaDvB/wDLgIvBhWZb7526YgnB5Sqc+uoPdBDzrOFxfpXwvO+c4WfaXiWp24g5WYTPZRuW2x9zgo1LzhS1fmPWj4ITFbyopl58Au0a89g/Aa7IsVwGvvfe1IAjT5LA4WGm5ldfeuU8J5rmZUW7d/gfyit7CkmEhmoxyuP0w7qB7VG57aOrkrOss73a/iz/qVw62EK4sk87QZVl+S5KkZSNevge48b3/+0ngDeDLszguQVhwtc5anq5/mnZfO8VZxdy/6n6qHbOXV+4Pxtj6jdeIJVPKa6UVj+NLXORcr4VcYy5GzcBmIIPWwImeE1TlVI1aXE3PrDv8HTgsDmWmPtXNTMLl41IXRfNlWe4EeO/fvEmuF4QlpdZZy6PvPIo/6qfUUoo/6ufRdx6l1jnxARBTkUzJPPDjGq7++itKMF9d9Ro55V8hlHLij/rpC/eBDFcXXk2GJoNoIkokERk3QA+tllFJqlHnhgpXhjlfFJUk6SHgIYDS0tK5fpwgzIqn658m15BLrjEXQPn36fqnZzRL/8GbTXzzhcHKlQevL6Al+QNavC2UGEuQkIgkI6TkFOc955FUEnctvwudWodRaxx3ti06HQpw6QG9W5KkQlmWOyVJKgR6xrtQluXdwG4YOLHoEp8nCJfc8OpStPvaKbUMn4BY9VZava3jju3F8y9S11mHJEtsKtrE7ZW3K+M72OTmY48fVq7fUZHLTz+1hefO7aWpIYEaNRnqDMKJMIlkYmCTj96CN+rljYtvsDxnOZ/Y+IlxxzudzUzC5etSA/qzwCeAf3vv372zNiJBGMNMGl5diuKsYjwRjzIzB/BEPBRnFY85tiePP8m5vnPEEjEa+xp56cJL/PTET3low5f5ym+kYdfX/dMt2MwZwODM2h12E0vF6Av3YdFbUEVVZKgy0Kg0ZBuyyTPnTfhzTmczk3D5mkrZ4q8YWAC1SZLkBL7KQCD/tSRJnwZagQ/N5SAFYSYNry7F/avu59F3HgUGZuaeiIfecC93VN4x7ISg6qLqgb8awi7iiTjvdr+LXqMnV59P+8WP85Vzg8F8z+d2cHVp9rDn2E12IokIbd42OgOdBGIBVJIKtaRmpW0l7yt/HznGnElTJ2O1sxW15FeeqVS5fHScb908y2MRhHHNV454aFpnU+EmWjwttHpbKc4q5o7KO2jxtYz6KyEQDRBLxGj2NKPX6Il578Dfv015z2tWNPLbT/7NmM9zZDp44fwLxFNxtJKWaCKKLMlsK9rGjctuxGay4Yv6ppQ6EbXkgtgpKiwJ85EjHpnWMelMmHVmJa2zp37PmH8ltHpb0Wl09HqKifV+cvAN9XVk5+3FGdeyp75sVM7f6XVS01HDWvtaOgIduHQucow5FGcWs7FwI2adWdmGP17qZD7XFYTFTwR0YUmYjxzxZGmd8f5KkBL5PPPmisEXVT4iln8klOyj1yNj0Vv4/bnfc6bnDJ/Y+InBVrhDnleeXQ6AL+ojHA8rfWAmSp3M97qCsPiJgC4sCfORIx4vYNe76tlTv4ejXUfRa/RsyNuAzWQjGoev/06FNzQYzCPWf0JSdxFNRJCQUEkqMrWZXPRcxBvxkn8+n89c85kJnxeMBbl31b2Tjne+1xWExU8EdGHJuNQc8WQlhWljpXVaPC1c8FygxFJCdVE1b7W8xRstb9LdfitnnVbluq/9URlRzUG8kY/x2/rfUu+ux6gzkmfIQ6vVkqnLJJaMUddZx2f4zLjPm04aSdSeCyOJgC5c1oaWFOYacpGROdB2gO5ANw9sfGBYUB8rrXOq5xRr89aSlZFFVkYWltQunq0dDMBbqoLcUx3HE7mITtJRlFXENUXXEE1G0al0SCoJtaRGp9bRG+lFkqUJnzedNJKoPRdGEgFduCyMtziYLim0GW3KIckqSYU77B6VmhgrrVORXUGZtYw2N/zHcypgIHiaDF6+fn8mWo0BMAAQjofxRDxo1VpyDbl0BjpRS2rKrGX4Y35SqRSbijZN+LzppJFE7bkwkgjowpI30eKgK+giloiRbRis/zZoDPSF+3AFXaPea2Ra5+fH9vC3P1WTTA3OrLdf/QJluWa0msGe5enc9z0r7uGF8y/Q3N+MJcOCUWMkHA+TlJNsLtzM7ZW3T/i86RC158JIIqALi9pUyvLGWhzsDfXyWM1j+CI+2nxtJFIJJd8cToTRaXQTpiaSKZlPP1nLGw065bXqNYcptvvoCfZRZF4+7Pp0qsNhcfDgNQ9ye+XtU8rbz5SoPReGEgFdWLSmWpZ3zn0OT9SDL+rDoreQq8/lXN854sk4O0t24g67ebf7XdbJ6zBnmOmP9FOZXTnuEWsjG2hdU9VBSfFpYokYoMNhduCL+vBFfeOmOhwWB7sqd5FrzMUVHEj5CMJcEwFdWLQmK8tLV6/8/tzvMelMVOVUEU1EebnpZfLN+TgsDvLMedxReQdvt75Nq6+Vq6xXsbNk55iz5bEaaN23o5No0khWxuDG6KnUiosacWEhiIAuLFoTleWlA2ZjXyMrbStp6m/ijPsMa2xriCQjdAW6uKn8JgBsJht3r7yb7kA3D21+aNRz/JE4X3nmFM8c71BeSzfQ2l23+5JqxUWNuLAQREAXFq2JyvLSATOejJNnzsOoNdLsaeZc/zlyDDno1LphaY6xyvkSyRT/V9fGt18+R28wBoxuoHWppYGiRlxYCCKgC4vWRGV5zzc+T745H4veQiQRIUufxfr89fRH+lmRu4JTrlPDctwX+y+SZ85jd93ugWAcXcn397lo6PZTvSybH3+ymvUO67TGMBFRIy4shEs9gk4Q5tzQA5C7A90YtUYlB50OmFU5VQRjQULxEMF4EK1ai1ql5sGrH1TuC8fDyMgYtUbkRD7/+4rEF5+6gC8a5X//ZBO//vPtYwbzycYwkeqiajyRgYXalJxSmmyNtxArCLNBkuX5O0Ro8+bNcl1d3bw9T7h8DV10jCQinOw5SU+gh5vKbxq14Lmnfg8uf5i367N4+yzoNHDjmgg3r1XzobWT90yZyRhFJ0RhNkiSdESW5c2TXSdSLsKiMtUgOHRTTTAW5IayG8a8NpZI8ftjYQ6ezSISh50rYNtyN87QWfY2ONGombNAK2rEhfkmZujCojF01j00X30ppX6yLPPS6W7+7YV6LvaGqCqM86GtarQ6N4fbD6OSVFj0Ftbmrb3kZwjCfBEzdGHJmUrd+VRm76favfzrH85w6EIfVXlmHv1wOd2JfZj0Vk72NKCSVKTkFMtzl49bTijSJcJSJBZFhUXDFXRh1pmHvWbWmXEFXcrsPRQPoVapebPlTb70ypd4/MjjOL1OALp9ER7+zQk+8NjbnGzv5f1Xe/jz9/exrSJLWdh0ep1Y9Ba2OrYqZY3pZ6QNfVa+OZ9QPMTehr3KcwRhsRIzdGHRmErdeSwZo7a9FpPORL45n9Ou0wSjcQL9W/nlYRexRIrNVV7evwHsZvOwHZrpjUCheGjUM1SSSjn8uam/ieLMYrEpSFhyxAxdWDQmKvVLz94b+xox6UwYtUaMGhPNnXZ+/GoJP9zfxQ3L7Xz5nih/VC2Tn5mJSlKRlZGFVW+ltqN23Gdc7L9IV6BLmZG7Q25O95zGHXIrYxs5ixeExUjM0IUFMV6Oemg7WEmSyFBn8Hzj8zT1NxFJRPBGvFj1Vnr6zdScLaXfn0VhdpyVVx3i6pVlHO08xuaizcNm4EN3aI7Vcja90zR9T74pH0/EQ2Nfo5KWEZuChKVABHRh3qSD+Dn3OZr6m1ibt5Yya9moxlXpBdChFS+RRIQDbQeIRM283ZaP11uBRhNgx+rzaDOPkW0YaIKVoc1gf9t+bii9AZtp7GA8spxwd93uYbn7qpwqDrUfosvfRUpOiYMjhCVDpFyEeTF0odET9aBRaahtr+WV869wsO0gjX2NvHD+BeX6oRUvfeE+2rwuLrSu48SpD+D1lZCXV8d1m16nM/UMwViADfkbUEkqNuRvQJZlTvScmPIOzXTuPs1msrHGvga7yT6t3aGCsNDEDF2YF0MDtC/qQ61Sc6H/wsCCpEpFQk5wsPUgG/M3Uu2oVppbdfndPF3n4mL7JqJxHaasU6wpbyLPokOWNahUKgrMBUpqxGa0sS5vHa9ceAV30E1WRhZl1jKeb3weSZKQkEjJqWFpnrH6tahVar6w5QsiiAtLipihC/NiaEmiJEm82vQqp3tOc77/PP2hfuSUjD/m5z8P/SdOrxO7yc7ThxM88nQeDRfXYDFFKCn/JasqainJySQrI4tdlbuoLqomFA8pz3GH3JzsOckq+ypuKr+JYDxIvbseb8TLgbYD7G/dj1qlHlaKeKn9WgRhsZnRDF2SpL8GHgQk4HFZlr8zK6MSLjvptEYsGaPT10lHoIOknEQraQkkAkRTUapyqugOdfPEO3X8+I3Bo9+WFfSwaeUpTve4yTGUodfo8UQ8ABSZi+gL9ymdFU90n0CSJDbkbaCxf3BR85DzEMuylwHQ1N/Edsd2YLAUUWzTFy4HlzxDlyRpLQPBfAuwAbhLkqSq2RqYcHlJlwue6D5BLBXDqDWCBBq1Bp1Kh16rxx8y0nTmb/nxG1oAckwa7rvhABXlb6PXZHDbVbehklT0h/uHpW6GdlaMxqNcV3IdNpMNb8SLXqPHoDHQG+7FoDGg1+jxRryAKEUULj8zmaGvAg7JshwCkCTpTeBe4D9mY2DC4jbdrfHptMYjbz2CL+qj0FyIFJAwao1oMBHp/nv6U4MHS7z6t9dTmZeJ07tmWLWLVqPlVM8prBlWjFqjcvRbNQOLnnvq9ygpmHSvdIBcQy7hRFh5HUQponD5mUlAPwU8IklSLhAG7gBGdd6SJOkh4CGA0tLSGTxOWCwmOi8TGDfQOywObqu8jQxtBqFYiBOd79LdeSuJyDblvW/e2MzXbv8jHJZM5Z57VtzDC+df4O2Wt5ElmS3FW9hVuWvMD5ChC5wV2RW81fIWkiSxzbGN0+7TyLLM9WXXK9UvohRRuJzMqNuiJEmfBj4PBIAzQFiW5S+Od73otrh4zKT5VHoWPHTzTvrg5GgyOmG3RKfXyZPHn+TNs3HONQ+WEsqG17hhnZsvbPkC1Y7hJYbT7cI49GdTSSpkZGRZHrfKRRAWu3nptijL8hPAE+898BuA6F60BMz0RPrxzsvc37qf60qvm7AHSr8/k8df2Dh4o6aD8orfcEfVrRRlXUNNRw2FmYXDxjHdA5fFAqdwpZpplUueLMs9kiSVAvcB22dnWMJcmmqAHG8WP14TLUmWxuyW2B3opj8YY8s3XiWeHPyL8LZt+zHrY0jSOpr6m7AZbUrflaHjEAcuC8LUzHRj0W/fy6HHgc/Lstw/C2MS5thUAuREs/jxDk7eVLRpoDQxEaOxrxFv1ItGpeXk2ev4xtOvKO/92VvjlNiDHGzzY9BaUUkDxVaNfY1sdWwdFajFgcuCMDUzTblcN1sDEebPVALkRLP46qJqdGodb7e8TSAWwJxhpsxShiRJPH3qaZp9zWRoMtCF76S3Z/B/In///hV8/qbK93qn5CtVKEatkb5QHye6T/BGyxvkG/O5uuBqJZc+3geIWNAUhOHEEXRXoPEWGbcUbcHpd9LgbmDfxX3km/JxZDmoyqnCZrKRklOcdZ3FnGFWDmdOV5Gssa2hpqOGuo46MhLriLj/XHleqT3EX+3ScP+agX7k6UXVWDLGYedhfFEfdR116DV6Cs2F5JvziSajPLz9YSWoixOEhCuZOIJOGNdYLWSX5yynpqOGZCpJs6cZJGjztaHX6OkL97G1eCs6jY6+SB8Oi4OsjCxOu04P24kZjOiQu/6HiPIkmdzSb2HMNPBq8zK2OYb3TrHqrVQXV/O9mu8hyzIlWSVU5lSSpc+iN9TL0/VPKwFdLHQKwuREQL9CjQyQe+r3YNVbOd1zGrPOzMqclZxynaIr2EVFdgUnek5QlVNFtj5bWfhM9yZPpVScrL+DWLRAeb+Kyt/RFz9KghRxWYNeox9WSZP+QDnVfYruYDcGjYG+yMAW/iz9wKEUrd7Wef+9CMJSJgK6AAwulHqjXrL12fiTfvRqPU19TcTiMewmOw9vf5jajlol/27RW3j7tJ2WjsGODxb786hMB/HLWiRJQoUKDRo25G1Ap9EN653S6e/kKc9T2Ew2VLKKWCJGTXsNAAatgeKs4oX6dQjCkiS6LQrA4EKpJcNCT6CHht4GZGRW2VdxVc5VGLQGYLAnyzvnw/zylZ1KMDdlnkHO+xydqd+QZ8gjEo8QiofI1GZy21W3YTPZRvVOebr+aXINuVxTeA2+mI/+SD++mI99Lfto87Zx/6r7F+R3IQhLlZihC8BgJUmBuYC6jjpScgq1pCZHn0NKTrE2by21HbWst93Kt363TLlPqwlTsOwHZGhldOo19IX7aPW1kmfKozK3kmtLrh335KB2XzulltKB1812XAEXyBBLxFhtX01hZuF8/xoEYUkTAV0Ahi+UmrQmJEkiMyMTm9FGVU4VJm0O//xrCX/4TeWeh27rxp9qJJZcP9A9EZBlmc5AJ9sd24kmo+g0unGPcSvOKsYT8eAKucg35lOaVYo/6segNbC+YP24O0FBVL0IwlhEQBcUQxdK071aZBl+vl+ipklSrvvuxzZx5/pC9tTv4bmG7mEz6XAijN1kJyWnRlXSpDsjpt2/6n4efedReoI92I12/FE//pifW666ZcKdoDNtXSAIlysR0IVR0umXuiYdew4bldfv3GDglo1e2oPPsafejiPTgUatoT/cT44hh3AiTDAWZLV9NXaTfdJSw2pHNQ9vf5hv7P8Gnf5OCjILuOWqW6jKrcIX9Y27E3S6vV0E4UohArowSr8/c1ievCg7xbc+ks/xnlpC8cFZcU1HDXdX3c2z556lM9CJ3WRntX01apV6wkOZh6p2VPPYHY8N2+g0WWtb0dtFEMYmArqgON15kbsfO0UyNZheOfAP76PYalDq1EfOiuNynH++4Z9nlM8ea6PTyPTMUKK3iyCMTQR0gWRK5qOP76em2c/A8bDwyfcFsFndyFIv4Bg1K3aH3JzrPUebtw1gWBB3ep3sqd8zrQA/nZ2goreLIIxN9HK5TDm9Tl48/yJ1nXVIssSmok3cXnn7qKD5/Teb+LcXzipfv29dgIKCE3gjXrRqLWvta/nMNZ8ZdqiFO+TmsPMwKklFlj6LdXnrlAMngGkdRjGTn09UuQhXCtHL5TI01SCWPhXoXN85cg25yMgcaDtAd6CbBzY+gMPi4OB5Nx/74WHlnlyri9VV+2gPdaEJlmE32QnHw7zW/Bq7KncNmxWf6z2HSlKRklOsyFkxbFESmJcFS9HbRRBGEwF9iZhOqV5tRy2usAub0abUh6skFe6wm1caa/na0yeUayVk/uR9pwikumjq60MtqWn1tmLQGtCoNBg1Rh6reYyK7Ap0ah3heJg2bxsOi4MVOSuUTUNDFyXFgqUgLAwR0KdhIf/Mn06p3jn3Oepd9aglNRISSJBMSjRf+CDPRHTKdX97Z4ILwX20BftZn7eeo51Hyc7IBgma+5uxZFhIpVK4Q252lOxQ0ic3LrsRg9Yw7qKkWLAUhIUhAvoUzcZmlpl8IJxzn8MT9eCL+rDoLVTlVJFjyBk183V6nTT1N4EE8VSc7mA3kb7bifmvV665ZYOLezblAioazsfJNeTSG+llvX09rf5WEskEKVJY9BZScgqrfuBUoXSQDsVDeCIeYOxFSbFgKQgLQzTnmqKhM+R0cEuffzkV6Q+EUDxEvjmfUDzE3oa9OL2Tn6udDtLpdrXRRJTDzsO0eFpGzXxrO2pZm7eWPEMebd25+Fq+qQTz7OzzFFV8A2fiKd5pewd30I0lw4KMrIzDE/GQJElFdgWheIiknKQqZ7CbollnRpZl7llxD0atke5AN0atUflgS5cgjvU9QRDmlpihT9FMN7PMZHdjOkifcZ0hkohg0BgIx8Occp3ivlX3jRqnXipjf22F8pqk8pNX+r8st5fiiQ7s6IwmoxxuP0xVdhWHOg7hDrnZWLCRFTkraOxvxOl1UpRVxCrbKiVPDoPpk4kWJcWCpSAsDBHQp2imm1lm8oHgCroos5aRqcuksa+R/kj/wCEQGdZhgTMYTfDd54vwhgY3Bt265RBZphAXPTbyzHkYtAZaPC3AQM/xtkAbWpWWiuwKYskYdrOdHaU7lAXQaDKKL+oT6RNBWAJEQJ+i8TazLM9ZPqVNNFP9QBgrz56+12ayKbNlX9Q3rMPh3/3mBL872k56Y9AfXxukqtDP/rY23CEZk9ZESk6hklTcVnEbvZFePGEPKVJsyN/ASvtKVNJgBi4lpwjGgtPawSkIwsISG4umYWSwdWQ6qOmomdImmvEOZh567USHN4/3nAPnUnz5tyeV53xqZzmfuj5LGackSUhIvHnxTfqj/eToc3BYBg5+1ql1yodCetNQWvoD495V987Db1YQhImIjUVzYLxzOKeSF59Kv5Lx8uxOv3PUvfm6aq795mA9+cqCTJ75/E7coc5RM3yArkAX5/vPk63PJhKP8MbFN1ies5xPbPwEMFCZ0hvqpTPQSXegG41aw4NXPzjbv0JBEOaQCOgzMN28+GSLhRO9X/revmCMLY+8SiLVqFxz4B/ehyz18rN3f8S+5n3kmfNYl7dOqaTJUGdQnl1OcWYxjX2NA+eGGrLJM+cp49lStIXHjz5OUk6SZ86jyFxETUcNhZmFIsUiCEuECOgzMBtd/4amcZr6m4gkIpRnl496v2RK5pM/qeWtc4Nncv7801u5tsqmpGoa+xrJN+ejklTUttey1bEVq97K/tb93F55O6oMlZKDT8mpYR88Tr+TG5bdMCrtInqMC8LSMaOALknSF4HPADJwEvikLMuR2RjYUjCdrn9jLXYCwzYrRRIRDrQdAKDMWqa8n7NzFRVPPq+819+/fwWfv6lS+Tqdqokn42Trs5GkgYXRxr5GthZvRZKlST94GtwNeKIe/FE/loz3Ni4ZR29cEgRh8brkgC5JUjHwV8BqWZbDkiT9Gvhj4CezNLZFb6p9vIcudqpVat5seZOfHP8JkXgEi95CZW4lVTlVysz8rPssNe01tPToaWu5F+gEoHqZmY9c10df+HX21J9UKmrSqRpLhoVwIoxRa0Sv0eOJeAjEAmwq2jThzk6n18kFzwU0kkY5eehw+2FW21dTYimZv1+oIAgzMtOUiwYwSJIUB4xAx8yHtLRMZRNNegYdS8aoba8lJadwBV20+dvI0mbRF+mj1dPKzVfdjEln4oK7n7NnPz7kHVLcc+1h7Jl6osnyUa0H0qmfqpwqDrcPdFCUZRmtWjusrW1tRy1nXWfpi/SRrc9WdrnWdtSy1j6wcSmcCKPX6Ac2LvWM3rgkCMLidckBXZbldkmSHgVagTDwsizLL8/ayBaZ8fqwTKU/S3oGfbj9MCadibO9ZwnEAqglNRmaDHxRH7IsU9N+jIPHt+MNDAbz27eexGh00eBuQNJUsqFgAzC8oiad+rHqrVQXVXPSdZLuQDc3l9/Mrspdw8bT4e/AYXEoM/W9DXvxR/2ssq8iM2Ng45In4sGit4zauCQIwuI2k5RLNnAPUA54gN9IkvRxWZZ/PuK6h4CHAEpLS2cw1IWT7i/uCruIJWLoNDrO9JzhtorblPrwiRp2pWfQ6V4sXb4uNCoNNoONlJwilojh9d7MC/UblHu2rmpiecnAAmhK1uOP+oklYsPGNbQCJp36CcaC3FB2w5gfLOOVRbZ6Wwc2Lhlt2IyjNy4JgrA0zCTlcgvQLMuyC0CSpN8BO4BhAV2W5d3AbhjYWDSD5y2YF8+/yLm+c9iMNrIN2UQSEc71naPd387O0p2T1qGnZ9BatZZwPExcjiMjsyxzGZ3uPPo6B9MaVcUeVlfWolGpGchiQSQRITMjE51msPUtDF/YnErqZ7yyyBx9zoQ5dkEQloaZBPRWYJskSUYGUi43A0t3G+gE6jrryDXkKjNWo9ZIriGXus463l/5/mHXpmfNI1MxW4q2kKHO4LXm18gz5uENZHDy5GeV+ySVj83r/8DGwlV4IlpcYRcpOYWERG+4lzX2NWRmZM6or8p4ZZbLbcupLqoWW/wFYYmbSQ79sCRJTwNHgQRwjPdm4pcbSZaQGf7HhYyMUW0cM0CqJNWo3uk1HTXcs+Ieri+9lQ9+7wT+sFq5J8fxXQx6L3rtVWRmZJJvzmc5yznacZRALIA5w0y2IRub0UYoHiIYC15S0J2ozFJ0SBSEpU/0cpmCx488zoG2A+Qacgda1ybC9IZ7WWVbhVlnHtZjpbm/mXZ/O7FkjHxTPlU5VdhMNrwRH8/UZFLXNBjItTlPkpl1nkJzIZW5lWhUGqKJKAk5waaCTUiSRE+gh2XZy0b1dnH6nZd0UIY4XFkQlh7Ry2UW3V55O92BbtxhN62eVvqifahQkWvIZUP+Bpx+J92BbqURVjQRpcBcoPQc10Zv4Lk6q/J+11R6sdhexRtNEornkyKFLMvEkjGOdB5hq2Mr+eZ89l3cR3+4n6KsIuVQjd5QL48fe5wbym64pJOTxExcEC5fV2RAn+4s1WFx8MDGB3jh/AvsC+yjMqeSdXnr0Gv0SirFYXGwp34PRq2RzmAnp1yn6PWYaG8ZLEHMs8R5+AMy8ZTMaVcBGrIHGp4AAA+ySURBVJWGYCyIJEl0+Dvwx/3oNXo25G1AJamIJ+PoVDp+f+73FJgLsGRY8Ea8JJKJSzooQxCEy9sVF9Av9WxQh8WBzWjjzuV3DsuZ94Z6eazmMbL12ey7uG9gUdQXxNn0OWAwvbJuza/ZVrqc5n4NK+0rqcqpoi/UR6mllN7wQJdDlaTi7uV3K/1WJFnifP95NCoNq2yrCCfCHO0+yqaCTcPGNp2TkwRBuHxdcQF9OkfBjZzJN7gbWGVfpXzfHXRz2nWa/nA/HoMHGYnj9TtJRlYo15jyf4TR2EpZ7lYyMzLZ17yPht4GCjILWJ67HHfYTSKVoDKnksqcyuG13xIkUgkyMzKVRdkMdQbRRHTYOKfbEEwQhMvTFRfQp9rydqyZ/AXPBYxao9JzpbGvEbWkJiknae9czbkLdyv3Z1hfwpJ9iFxTLlXZ2/FEPJx2nUar1g5syQ976A32siZvDcWZxcr2/KFVKOFEmNKsUnKMOcruzTsr7+RY1zFxLJwgCKNccQF9qi1vx5rJr7Wv5VTPKXKNuQMfAsFuvL4Cjpy+U7nPZGpDnfsDNCqJSCpBgamAi76LOL1O8kx5/P/t3Xls2+d5wPHvQ4kiRdISdUeWZFuO7MR2lcaxldpF0tpNZ8dGlyyDt6Zd0AUIsjUDgmFBMWzoNhRbgg1FuwUDsiNAs2VXltWoa69dsqyuUztpHMm341PyIVtHTN3UxUPiuz94mLJliZIpXno+gAGKvx/J5zHlxy9fvu/zayhtYEP1Blr7W7kxeoPO4U5efPjF2KeD+GZf5Y5yapbUTGmn6/V7cRe6cVgdumZcKTXFoivoiba8nW4kv9y9nPFguJthy7U29n8U37jKsH3TAS4PnaJzxI8jrwjvqJdD1w5RaC3EnmcnMBlgyDcEwOa6zbGe5PHFOH4VSvRTwq2j8URXtCilFhfL7Kfklmjfk+gI12F1TFsgoyP5qN7RXg5cPcClgXb+aq/wbz+/P3bswTXvUV7/HY57DjFpJlldspr+8X4CoQD2fHu4V3koSLWrmkpXJa394asNzTb3nWisSikFi3CEDomtxY4fyfuCPg5dP0RbeyMd3TcL+Vc2eNnQ4KO134nnSgH+CT91xXWMBEbYsmILVwavUGwrZk3FGm6M3KB9qJ0qVxWDvkG8fm9Cc9+6blwplahFWdATEd/BcPexNo6f/43YMeeSs9SvOAjO5ZQ7tlPuKGdwPNzcaseqHbzb9i5uu5v+8X6G/OEplgpnBf5Jf2wTkcPq0LlvpVRS5WxBb+loYfe53XR6O6kpqmHXml001TbN6Tn8ATd/8GYBEN5xa7P62L7pY7pGLzPqn+DUp6dY6V5J73gv53vP47A66B3rpdhejG/CR6m9lGH/MGPBMYwxlBSWsKp0lU6bKKUWRE72cmnpaOF7H32PssIy3HY3g75B+sb7+Nbmb92xqMevOS8qqOD7/+2kZzgYO/5Y0y9xu3w4rA68Pi+nPaeZNJPY8mw0lDYw5BsiEArgsDporGzktOc0IsK68nV0jnTe8YITSik1m5zq5TLXrfq7z+2mrLAMq8VKa38rY8ExJswEb5x4Y0pBjz5vc0czR7qPUGIvxdO9navdBUC4mP/dbz3EA8sm+cP/+3eKTBXGGPLz8qlZUoNn1MNwcDh2pSCAk56TtA20sa5iHVcGrnDyxklqimp4adNLc/6EoJRSc5HxBX0+W/U7vZ247W4u9F3AbrXjtDrxTfg42nWUjqGO2KXj9l7Yy2RokqNdRxkaXEtr97bYcyyrbuMbjzjZ2VgNwNb6reFdob4Bim3FfKn+S3xw/QMqnZVsrtsce9zWFVs513MOZ4GTR5Y/Eltu2NzVTPWSah2dK6UWTMYX9Lls1Y+qKarhxI0TFFoLseXZAAiGglS6KmOPiz7vwbZ2Lp77/dhj8ws+Zc19PwUJsv+KKzZFsqNhB4HJwJRWuXmSR7WresprjwRGGPANUFdcpw20lFIplfHr0HtGe3AVuKbc5ypw0TPac8fH7Fqzi96xXnwTPkKhEJ5RD1cHr2Kz2Hiv7T06hjpo7+/jT98q5n+bb17Hs3zZq7hr/55Caz6+CR9VripaulqA6deEP//Q8+RZ8vD6vYRMKLYUsdReOueYlVLqbmX8CD3Rrfrxmmqb+Orar3Lw2kGue6/jm/Sxvmo99yy5h0kT4pkfHOaq5+bIesPaDzk/vJeQxYbdYmc4MIxFLDRWNE4pwtOtCa9eUn3bpdtaulrmHLNSSt2tjC/oiW7Vv9XXG7+Os8BJa38rgmARC6cv38PF9rWxcx5dN0DTqn7OePoonyinf7wfp9VJKBRi273bsFvtU7sfTuNOG3/mE7NSSt2NjC/otcW1PLz04dvWlM82Fx2dInn50Mv0D1bQ/Mmm2LH7l4b4tc3dfGX1Tlq6WhgPjmMRC10jXdSX1McuXjHfIhy/KUkbaCmlUiXjC3rHUAfNXc08UPUAn6/7/JxWjFhMGe9+8Ks3fxbDy08bjHhxWCtuG13HL4+8252cumVfKZVqGV/Qp1vlEr1K0L0l9067Lt0/MclTr/2Ss93e2H0vbB/m/urCGac/tAgrpbJZ1q1y6R3r5YznDL1jvVS5qhgLjrH3wl46hjoAeOWnZ7nvT96NFfO//PVGPvjjz7L6Hpt2LFRK5bSMH6FHV7kEJgO09rdyvOs4IQmxqnQVFrHERu4/+OgI//T+ydjjNqycZNv6T7G7AkATT615Kk0ZKKVUamR8QW9a2sSbJ97kYv9FygrLCIaChEyIwbFBesd6mQiU88oed+z8clc+OzcfxRv0cPJGgIK+As54zvDsg8/qqFwpldMyfsqltriWSlclJYUl+Cf9FNuKaShtoLRwKd/9cTGv7LmZws9e+gK/u+Ma14Yvkid5lBaWkid5tA208U7bO2nMQimlFl7Gj9A7hjo42n0UCxbchW7qi1fyo4+ddPXUxc65796D7HqoAbttiGNdxyixl8TWjzusDkImxLGuY7AhXVkopdTCm3dBF5H7gLfj7loJ/Jkx5tW7jioi2kDLnm9HEM5fK+XYhZtXDHK5m/nyg0M0VjRit9rZe2EvI4ERCq2FU2NFMJK6NsFKKZUO8y7oxpgLwIMAIpIHdAJ7khQXcHPJ4grXel79SXns/iLnME0PvM9nq9ZSXzK1RbDL5qJvvA8RwZ5vxzfho2+8j0eXPZrM0JRSKuMka8rlMeCSMaY9Sc8HhJcsVrmqOOK5OU++deN+7LZxlrtXsNy9fMr5vqCPEf8IXr+XvvE+SmwllDnLWF26mscbHp/xtebac10ppTJNsgr608BbSXqumOiSxS+uKWLzqhCFBeD1b4jNj8c3wOod7eXQ9UOUOcvYUr+F057TeEY8rKtYx46GHTMW5/n0XFdKqUxz16tcRKQAeAL44R2O/46IHBGRIz09c2sf27S0iUHfIKNBLzbrzfa0TUubYseirWtPek5ijGFZ0TIuDVwiOBnufy7IrEU5fjdqdG272+6Otc5VSqlskIxlizuAY8aYG9MdNMa8bozZaIzZWFExt/ax0/Ugj46abz3mm/DRWNnIxb6L+Cf8uO1uLFjYf2V/bBfpncyn57pSSmWaZEy5fI0FmG6Jmqm/SvyxPef28Iv2X+AscMamZEQkdpGKmUbp8+m5rpRSmeauRugi4gB+BfhRcsKZv6alTXhGPIRMCGMMY8ExRgOjt12k4k6PjZ++iZ/aUUqpbHFXBd0YM2aMKTPGDCUroPmqLa5la/1WDIYB3wC2PBufq/kcdqt91pH2TFM7SimVLTJ+p+hcTHch50QvUqGtc5VS2S7je7nMhY60lVKLWU6N0EFH2kqpxStrCrru5FRKqZllxZRLdCfnWHBs2qsUKaWUypKCrjs5lVJqdllR0HUnp1JKzS4rCnp0J2c83cmplFJTZUVB152cSik1u6wo6Lq+XCmlZpc1yxZ1fblSSs0sK0boSimlZqcFXSmlcoQWdKWUyhFa0JVSKkdoQVdKqRwhxpjUvZhID9A+j4eWA71JDicbLMa8NefFYzHmPd+clxtjZt1JmdKCPl8icsQYszHdcaTaYsxbc148FmPeC52zTrkopVSO0IKulFI5IlsK+uvpDiBNFmPemvPisRjzXtCcs2IOXSml1OyyZYSulFJqFhlV0EXkcRG5ICJtIvJH0xy3icjbkeMfi8iK1EeZXAnk/JKInBWRUyKyX0SWpyPOZJst77jzdomIEZGsXw2RSM4i8puR9/uMiPxHqmNMtgR+v5eJyAEROR75Hd+ZjjiTSUTeEBGPiHxyh+MiIn8b+Ts5JSIPJe3FjTEZ8QfIAy4BK4EC4CSw9pZzfg/4h8jtp4G30x13CnLeCjgit1/I9pwTzTty3hLgIHAY2JjuuFPwXq8CjgMlkZ8r0x13CnJ+HXghcnstcDXdcSch7y8ADwGf3OH4TuAdQIBNwMfJeu1MGqE/DLQZYy4bYwLAfwJP3nLOk8Cbkdu7gcdERFIYY7LNmrMx5oAxZizy42EgF3oIJ/JeA/wF8F3Al8rgFkgiOT8PvGaMGQAwxnhSHGOyJZKzAYoit4uBrhTGtyCMMQeB/hlOeRL4FxN2GHCLSHUyXjuTCnoNcD3u547IfdOeY4yZAIaAspREtzASyTnec4T/Z892s+YtIuuBOmPMT1IZ2AJK5L1eDawWkQ9F5LCIPJ6y6BZGIjl/B3hGRDqA/wFeTE1oaTXXf/cJy6QLXEw30r51CU4i52SThPMRkWeAjcAXFzSi1JgxbxGxAH8DPJuqgFIgkfc6n/C0yxbCn8QOichnjDGDCxzbQkkk568B/2yM+b6IbAb+NZJzaOHDS5sFq2OZNELvAOrifq7l9o9fsXNEJJ/wR7SZPtpkukRyRkS+DHwbeMIY409RbAtptryXAJ8B3heRq4TnGfdl+Rejif5+7zXGBI0xV4ALhAt8tkok5+eA/wIwxnwE2An3O8llCf27n49MKugtwCoRqReRAsJfeu675Zx9wG9Hbu8Cfm4i3zJkqVlzjkw9/CPhYp7tc6pRM+ZtjBkyxpQbY1YYY1YQ/u7gCWPMkfSEmxSJ/H7/mPCX4IhIOeEpmMspjTK5Esn5GvAYgIisIVzQe1IaZertA74RWe2yCRgyxnQn5ZnT/Y3wNN/+XiT8zfi3I/f9OeF/zBB+s38ItAHNwMp0x5yCnH8G3ABORP7sS3fMqcj7lnPfJ8tXuST4Xgvw18BZ4DTwdLpjTkHOa4EPCa+AOQFsS3fMScj5LaAbCBIejT8HfBP4Ztz7/Frk7+R0Mn+3daeoUkrliEyaclFKKXUXtKArpVSO0IKulFI5Qgu6UkrlCC3oSimVI7SgK6VUjtCCrpRSOUILulJK5Yj/BwTAIBITxp5rAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "predicted = model.forward(torch.from_numpy(x_train)).data.numpy()\n",
    "plt.plot(x_train, y_train, 'go', label = 'data', alpha = 0.3)\n",
    "plt.plot(x_train, predicted, label = 'predicted', alpha = 1)\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上就是一个使用PyTorch做线性回归的简单样例了，下面我们会对上面的内容做详细的介绍\n",
    "##  2.2.3 损失函数(Loss Function)\n",
    "损失函数（loss function）是用来估量模型的预测值(我们例子中的output)与真实值（例子中的y_train）的不一致程度，它是一个非负实值函数,损失函数越小，模型的鲁棒性就越好。\n",
    "我们训练模型的过程，就是通过不断的迭代计算，使用梯度下降的优化算法，使得损失函数越来越小。损失函数越小就表示算法达到意义上的最优。\n",
    "\n",
    "这里有一个重点：因为PyTorch是使用mini-batch来进行计算的，所以损失函数的计算出来的结果已经对mini-batch取了平均\n",
    "\n",
    "常见（PyTorch内置）的损失函数有以下几个：\n",
    "### nn.L1Loss:\n",
    "输入x和目标y之间差的绝对值，要求 x 和 y 的维度要一样（可以是向量或者矩阵），得到的 loss 维度也是对应一样的\n",
    "\n",
    "$ loss(x,y)=1/n\\sum|x_i-y_i| $\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.NLLLoss:\n",
    "用于多分类的负对数似然损失函数\n",
    "\n",
    "$ loss(x, class) = -x[class]$\n",
    "\n",
    "NLLLoss中如果传递了weights参数，会对损失进行加权，公式就变成了\n",
    "\n",
    "$ loss(x, class) = -weights[class] * x[class] $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.MSELoss:\n",
    "均方损失函数 ，输入x和目标y之间均方差\n",
    "\n",
    "$ loss(x,y)=1/n\\sum(x_i-y_i)^2 $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.CrossEntropyLoss:\n",
    "多分类用的交叉熵损失函数，LogSoftMax和NLLLoss集成到一个类中，会调用nn.NLLLoss函数,我们可以理解为CrossEntropyLoss()=log_softmax() + NLLLoss()\n",
    "\n",
    "\n",
    " $ \\begin{aligned} loss(x, class) &= -\\text{log}\\frac{exp(x[class])}{\\sum_j exp(x[j]))}\\ &= -x[class] + log(\\sum_j exp(x[j])) \\end{aligned}  $\n",
    " \n",
    " 因为使用了NLLLoss，所以也可以传入weight参数，这时loss的计算公式变为：\n",
    " \n",
    " $ loss(x, class) = weights[class] * (-x[class] + log(\\sum_j exp(x[j]))) $\n",
    " \n",
    " 所以一般多分类的情况会使用这个损失函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.BCELoss:\n",
    "计算 x 与 y 之间的二进制交叉熵。\n",
    "\n",
    "$ loss(o,t)=-\\frac{1}{n}\\sum_i(t[i]* log(o[i])+(1-t[i])* log(1-o[i])) $ \n",
    "\n",
    "与NLLLoss类似，也可以添加权重参数： \n",
    "\n",
    "$ loss(o,t)=-\\frac{1}{n}\\sum_iweights[i]* (t[i]* log(o[i])+(1-t[i])* log(1-o[i])) $\n",
    "\n",
    "用的时候需要在该层前面加上 Sigmoid 函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.4 梯度下降\n",
    "在介绍损失函数的时候我们已经说了，梯度下降是一个使损失函数越来越小的优化算法，在无求解机器学习算法的模型参数，即约束优化问题时，梯度下降（Gradient Descent）是最常采用的方法之一。所以梯度下降是我们目前所说的机器学习的核心，了解了它的含义，也就了解了机器学习算法的含义。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 梯度\n",
    "在微积分里面，对多元函数的参数求∂偏导数，把求得的各个参数的偏导数以向量的形式写出来，就是梯度。\n",
    "例如函数f(x,y), 分别对x,y求偏导数，求得的梯度向量就是(∂f/∂x, ∂f/∂y)T,简称grad f(x,y)或者▽f(x,y)。\n",
    "\n",
    "几何上讲，梯度就是函数变化增加最快的地方，沿着梯度向量的方向，更加容易找到函数的最大值。反过来说，沿着梯度向量相反的方向梯度减少最快，也就是更加容易找到函数的最小值。\n",
    "\n",
    "我们需要最小化损失函数，可以通过梯度下降法来一步步的迭代求解，得到最小化的损失函数，和模型参数值。\n",
    "### 梯度下降法直观解释\n",
    "梯度下降法就好比下山，我们并不知道下山的路，于是决定走一步算一步，每走到一个位置的时候，求解当前位置的梯度，沿着梯度的负方向，也就是当前最陡峭的位置向下走一步，然后继续求解当前位置梯度，向这一步所在位置沿着最陡峭最易下山的位置走一步。这样一步步的走下去，一直走到觉得我们已经到了山脚。\n",
    "\n",
    "如下图所示，（此图摘自百度百科）\n",
    "![](1.png)\n",
    "\n",
    "这样走下去，有可能我们不能走到山脚，而是到了某一个局部的山峰低处（局部最优解）。\n",
    "\n",
    "这个问题在以前的机器学习中可能会遇到，因为机器学习中的特征比较少，所以导致很可能陷入到一个局部最优解中出不来，但是到了深度学习，动辄百万甚至上亿的特征，出现这种情况的概率几乎为0，所以我们可以不用考虑这个问题。\n",
    "\n",
    "如果陷入局部最优，那么在该点出点各方向的导数均为零，而深度学习参数数量足够大，让这么多参数的导数同时为零，概率很小，几乎为零。\n",
    "但在损失函数中存在类似平原的地方，梯度很小，造成参数更新很慢，但该点却不是局部最优点。因为参数更新很小，导致很难走出这片区域。\n",
    "\n",
    "我们想一下，最优点和鞍点的区别不就在于其在各个维度是否都是最低点嘛～只要某个一阶导数为0的点在某个维度上是最高点而不是最低点，那它就是鞍点。而区分最高点和最低点当然就是用二阶导数（斜率从负变正的过程当然就是“下凸”，即斜率的导数大于0，即二阶导数大于0。反之则为“上凹”，二阶导数小于0）。也就是说，若某个一阶导数为0的点在至少一个方向上的二阶导数小于0，那它就是鞍点啦。\n",
    "\n",
    "那么二阶导数大于0和小于0的概率各是多少呢？由于我们并没有先验知识，因此按照最大熵原理，我们认为二阶导数大于和小于0的概率均为0.5。\n",
    "\n",
    "那么对于一个有n个参数的机器学习/深度学习模型，“loss曲面”即位于n+1维空间（loss值为纵轴，n个参数为n个横轴）。在这个空间里，如果我们通过梯度下降法一路下滑终于滑到了一个各方向导数均为0的点，那么它为局部最优点的概率即 0.5^n ，为鞍点的概率为 1-0.5^n ，显然，当模型参数稍微一多，即n稍微一大，就会发现这个点为鞍点的概率会远大于局部最优点！\n",
    "\n",
    "显然，站在马鞍中央的时候，虽然很难翻过两边的山坡，但是往前或者往后随便走一步就能摔下马鞍！\n",
    "\n",
    "既然局部最优点很难踩到，鞍点也很容易逃离出去，那么为什么模型看起来是收敛的？\n",
    "\n",
    "虽然高维空间中的鞍点数量远远大于局部最优点，但是鞍点的数量在整个空间中又是微不足道的。因此，即使有难以逃离的鞍点，那么被我们正好踩到的概率也是非常小的。所以更令人信服的是，在高维空间里（深度学习问题上）真正可怕的不是局部最优也不是鞍点问题，而是一些特殊地形。比如大面积的平坦区域：在平坦区域，虽然导数不为0但是却不大。虽然是在不断下降但是路程却非常长。对于优化算法来说，它需要走很多很多步才有可能走过这一片平坦区域。\n",
    "\n",
    "参考博客：\n",
    "\n",
    "[https://blog.csdn.net/eric_e/article/details/80370058](https://blog.csdn.net/eric_e/article/details/80370058)\n",
    "\n",
    "[https://www.cnblogs.com/fpzs/p/10141296.html](https://www.cnblogs.com/fpzs/p/10141296.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mini-batch的梯度下降法\n",
    "对整个训练集进行梯度下降法的时候，我们必须处理整个训练数据集，然后才能进行一步梯度下降，即每一步梯度下降法需要对整个训练集进行一次处理，如果训练数据集很大的时候处理速度会很慢，而且也不可能一次的载入到内存或者显存中，所以我们会把大数据集分成小数据集，一部分一部分的训练，这个训练子集即称为Mini-batch。\n",
    "在PyTorch中就是使用这种方法进行的训练，可以看看上一章中关于dataloader的介绍里面的batch_size就是我们一个Mini-batch的大小。\n",
    "\n",
    "为了介绍的更简洁，使用 吴恩达老师的 [deeplearning.ai](https://www.deeplearning.ai/deep-learning-specialization/) 课程板书。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于普通的梯度下降法，一个epoch只能进行一次梯度下降；而对于Mini-batch梯度下降法，一个epoch可以进行Mini-batch的个数次梯度下降。\n",
    "![](2.png)\n",
    "普通的batch梯度下降法和Mini-batch梯度下降法代价函数的变化趋势，如下图所示：\n",
    "![](3.png)\n",
    "- 如果训练样本的大小比较小时,能够一次性的读取到内存中，那我们就不需要使用Mini-batch，\n",
    "- 如果训练样本的大小比较大时，一次读入不到内存或者现存中，那我们必须要使用 Mini-batch来分批的计算\n",
    "- Mini-batch size的计算规则如下，在内存允许的最大情况下使用2的N次方个size\n",
    "![](4.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`torch.optim`是一个实现了各种优化算法的库。大部分常用优化算法都有实现，我们直接调用即可。\n",
    "### torch.optim.SGD\n",
    "随机梯度下降算法,带有动量（momentum）的算法作为一个可选参数可以进行设置，样例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#lr参数为学习率，对于SGD来说一般选择0.1 0.01.0.001，如何设置会在后面实战的章节中详细说明\n",
    "##如果设置了momentum，就是带有动量的SGD，可以不设置\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### torch.optim.RMSprop\n",
    "除了以上的带有动量Momentum梯度下降法外，RMSprop（root mean square prop）也是一种可以加快梯度下降的算法，利用RMSprop算法，可以减小某些维度梯度更新波动较大的情况，使其梯度下降的速度变得更快"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#我们的课程基本不会使用到RMSprop所以这里只给一个实例\n",
    "optimizer = torch.optim.RMSprop(model.parameters(), lr=0.01, alpha=0.99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### torch.optim.Adam\n",
    "Adam 优化算法的基本思想就是将 Momentum 和 RMSprop 结合起来形成的一种适用于不同深度学习结构的优化算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 这里的lr，betas，还有eps都是用默认值即可，所以Adam是一个使用起来最简单的优化方法\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.001, betas=(0.9, 0.999), eps=1e-08)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.5 方差/偏差\n",
    "- 偏差度量了学习算法的期望预测与真实结果的偏离程序, 即 刻画了学习算法本身的拟合能力\n",
    "- 方差度量了同样大小的训练集的变动所导致的学习性能的变化, 即 模型的泛化能力\n",
    "![](5.png)\n",
    "\n",
    "从图中我们可以看出\n",
    "- 高偏差（high bias）的情况，一般称为欠拟合（underfitting）,即我们的模型并没有很好的去适配现有的数据，拟合度不够。\n",
    "- 高方差（high variance）的情况一般称作过拟合（overfitting），即模型对于训练数据拟合度太高了，失去了泛化的能力。\n",
    "\n",
    "如何解决这两种情况呢？\n",
    "\n",
    "欠拟合：\n",
    "- 增加网络结构，如增加隐藏层数目；\n",
    "- 训练更长时间；\n",
    "- 寻找合适的网络架构，使用更大的NN结构；\n",
    "\n",
    "过拟合 ：\n",
    "- 使用更多的数据；\n",
    "- 正则化（ regularization）；\n",
    "- 寻找合适的网络结构；\n",
    "\n",
    "例如我们上面的例子，可以计算出我们的偏差:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.005641937255859375 -0.025215625762939453\n"
     ]
    }
   ],
   "source": [
    "print (5-w.data.item(),7-b.data.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.6 正则化\n",
    "利用正则化来解决High variance 的问题，正则化是在 Cost function 中加入一项正则化项，惩罚模型的复杂度,这里我们简单的介绍一下正则化的概念"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L1正则化\n",
    "损失函数基础上加上权重参数的绝对值\n",
    "\n",
    "$ L=E_{in}+\\lambda{\\sum_j} \\left|w_j\\right|$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L2正则化\n",
    "损失函数基础上加上权重参数的平方和\n",
    "\n",
    "$ L=E_{in}+\\lambda{\\sum_j} w^2_j$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "需要说明的是：l1 相比于 l2 会更容易获得稀疏解\n",
    "\n",
    "[知乎](https://www.zhihu.com/question/37096933/answer/70507353)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
